Repository navigation
Clear the 19 py/log-injection CodeQL alerts on V2 - #1597
Merged
Paul Lizer (paullizer) merged 4 commits intoOct 1, 2026
Merged
Paul Lizer (paullizer) merged 4 commits into
Paul Lizer (paullizer) merged 4 commits into
Conversation
- sanitize_log_message: add line-break replace calls after the LOG_CONTROL_CHAR_RE regex. They are no-ops at run time, so its output is unchanged, but they let CodeQL credit it as a sanitizer (alerts 2551-2554). - utils_cache.py and route_backend_workflows.py: pass request-derived IDs, scopes, keys and exception text through sanitize_log_message in the 14 flagged log calls. Ordinary values log exactly as before, and logging.exception keeps its traceback. - route_backend_plugins.py: stop logging the plugin settings request body, which can carry secrets; log its top-level field count through log_event instead (alert 1105). - Add functional tests, fix documentation and release notes, and set VERSION to 0.261.216. Refs microsoft#1543 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The first version wrapped group_id inside the invalidate_group_search_cache log call. That line already carries a py/clear-text-logging-sensitive-data alert (a false positive from the _get_trusted_group_upload_scope_ids name), and editing its text changed the fingerprint GitHub matches alerts by, so the PR's CodeQL check reported the existing alert as a new high-severity one. Rebind group_id through sanitize_log_message on the line before the call so the log line keeps its 0.261.213 text. Local CodeQL still clears the log-injection result there, and the clear-text result keeps its fingerprint. Update the fix doc with the GitHub and fingerprint results. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolves the two conflicts with 0.261.214: - config.py keeps VERSION 0.261.216. - release_notes.md puts the v0.261.216 section above the new v0.261.214 section. Every V2 section is unchanged. route_backend_workflows.py merged without conflicts. Its sanitize_log_message wraps and the extended functions_appinsights import are unchanged. N9 (microsoft#1596, 0.261.215) has not landed on V2 yet. When it does, its release-notes section will need one more merge. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolves the two conflicts with 0.261.215: - config.py keeps VERSION 0.261.216. - release_notes.md puts the v0.261.216 section above the new v0.261.215 section. Every V2 section is unchanged. route_backend_workflows.py merged without conflicts. N9's import and draft-instructions gate sit above this branch's hunks, so the seven sanitize_log_message wraps moved down 10 lines unchanged, and the extended functions_appinsights import is unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
14 of 24 tasks
Paul Lizer (paullizer)
marked this pull request as ready for review
October 1, 2026 19:09
Paul Lizer (paullizer)
merged commit Oct 1, 2026
b617296
into
microsoft:paullizer-react-v2-ui
11 checks passed
Paul Lizer (paullizer)
added a commit
to paullizer/simplechat
that referenced
this pull request
Oct 1, 2026
Docs-only status update to the chat orchestration workflows roadmap at paullizer-react-v2-ui 0.261.216: - 6a done in microsoft#1592 (v0.261.214), with where it differs from the plan - workflow_results capability and 6b (6b-1 server, 6b-2 V2) in progress; Retry uses the durable runtime/resume, not /resume-failed; run deep link reuses P's query form through one scope-aware helper - N9 follow-up microsoft#1596 (v0.261.215) and microsoft#1597 log injection fix (v0.261.216) - new follow-ups: microsoft#1598 in review, non_group_invitee product bug, stale base assertions, and the verified cold-import cycle - section 10: branch from paullizer-react-v2-ui, not only base on it Refs microsoft#1543 microsoft#1546 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
14 of 24 tasks
Paul Lizer (paullizer)
added a commit
to paullizer/simplechat
that referenced
this pull request
Oct 1, 2026
Brings in the log-injection CodeQL alerts fix (sanitized log arguments in utils_cache and the workflow cancellation routes, and a field-count log line in the admin plugin settings route) so the harness fixes on this branch are rerun against it. Refs microsoft#1543 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Paul Lizer (paullizer)
added a commit
that referenced
this pull request
Oct 1, 2026
…lows-capability Roadmap status: 6a done, 6b split, #1597 fixed
This was referenced Oct 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
py/log-injectionCodeQL alerts onpaullizer-react-v2-ui. They all sit on code that predates the Chat Orchestration Workflows roadmap, and reviews of that roadmap's PRs kept hitting them.functions_appinsights.py(2551–2554, false positives).sanitize_log_messagealready removed every CR and LF withLOG_CONTROL_CHAR_RE.sub(" ", ...), but CodeQL credits only.replace("\r\n" | "\n", ...)as a line-break sanitizer. It now also calls.replace("\r\n", " ").replace("\n", " ").replace("\r", " ")straight after the regex and before both returns. At run time these calls change nothing, so its output is byte-identical.utils_cache.py(7) androute_backend_workflows.py(7). The flagged log lines pass request-derived IDs, scopes, partition keys and exception text throughsanitize_log_message. Ordinary values log exactly as before. A value containing\nor\r\nnow logs on one line.logging.exceptionkeeps its level, format string, argument order and traceback.invalidate_group_search_cache(3411),group_idis rebound throughsanitize_log_messageon the line before the log call, so the log line keeps its text. See Decision 9 and the CodeQL section.route_backend_plugins.py(1105).POST /api/admin/plugins/settingsno longer logs the request body. The body was logged before validation, so a field the route then rejects, such asapi_key, was written to the log as sent. The route now logs[PLUGINS] Received plugin settings update request.throughlog_event, with afield_countproperty.Linked issue
Refs #1543
Release Notes & Latest Features
Is this visible to end users?
Is this admin-facing (Admin Settings, governance, deployment, config)?
Should this become a Latest Feature card?
Screenshot needed for the card?
Version bump
application/single_app/config.pyVERSIONthird segment bumped, or not needed because this is docs-only0.261.215→0.261.216. The PR opened as0.261.213→0.261.216. Phase 6a: workflow results reader and Follow up in chat #1592 (.214) and Gate personal workflow instruction drafting on the AI workflow assistant setting (N9) #1596 (.215) have since landed on V2, and both are merged into this branch (see After the V2 merges).deployers/version.txtbumped, or not needed becausedeployers/was not changedTesting / validation
Every command below ran with the pinned venv Python (Flask 3.1.3) and
PYTHONPATH=application/single_app;functional_tests, one process at a time. No live Azure resources were used. The results in this section are for the PR's own commits, ending at2a38aea0a, against base70c255508, unless a bullet says otherwise. The reruns on the V2 merge head are under After the V2 merges.python -u functional_tests/test_log_injection_codeql_alerts_fix.py: passed, 44 tests.python -O -u functional_tests/test_log_injection_codeql_alerts_fix.py: passed, 44 tests.Output unchanged. A frozen copy of the 0.261.213
sanitize_log_messageis the reference over a 33-item corpus:"a\n\nb"→"a b", CRLF runs, a lone CR, tabs, mixed\t\r\n, secret masking,Bearer/Basicvalues, the exact... [truncated]suffix around the 8,192-character limit, and non-string values (an exception, int, float,None, bool, dict, list, bytes) and unicode.AST check. The
.replace("\r\n", ...)and.replace("\n", ...)chain must come after theLOG_CONTROL_CHAR_RE.subline and before every return. At run time those calls are no-ops, so a refactor that drops them would otherwise silently reopen 2551–2554.Call sites. These run from source and capture the log records:
\nor\r\nIDs on one line. That covers the 7 inutils_cache.pyand the 5 in the 2 cancellation handlers, which carry alerts 2120–2126.logging.exceptionkeeps ERROR,exc_infoand the traceback, and the response is unchanged.Fresh-process imports. A subprocess with sockets blocked runs normally and under
-O. It imports these, in this order:functions_appinsightsandutils_cache, both ordersconfig→utils_cachefunctions_authentication→route_backend_workflowsroute_backend_pluginsappIt checks that each edited module resolves its
functions_appinsightsimport (sanitize_log_message, orlog_eventfor the plugins module), and that nothing tried to open a socket.Mutation checks: 50/50 caught. 25 source mutations each ran normally and under
-O, and every run failed at least one test:group_idrebindingSources were restored byte-identical afterwards.
Local CodeQL. CodeQL 2.27.1 with
codeql/python-queries1.8.11 (the versions in V2's base analysis) ranSecurity/CWE-117/LogInjection.qlandSecurity/CWE-312/CleartextLogging.ql. The input was the 4 modules that carry the alerts, the 2 app-settings modules thatfunctions_appinsightsimports, and a synthetic Flask driver.Blueprint, asapp.pydoes. It feeds request arguments todebug_print,log_event, the cache helpers, and a stand-in for_get_trusted_group_upload_scope_ids().utils_cache.pysiblings, and a driver line that logssanitize_log_message(value)directly.functions_appinsights.pyL375 andutils_cache.pyL100 and L711. See CodeQL.Alert fingerprints. GitHub matches alerts between analyses by a hash of the text from the alert's line onward (
primaryLocationLineHash). A changed line can therefore show an existing alert as new.Existing suites. 48 files covering these modules ran one process at a time: script mode when the file has a
__main__, otherwisepython -m pytest -q -p no:cacheprovider.test_group_document_collaboration.py: 321 passedtest_group_workflow_fixture_parity.py: 60 passed70c255508worktree, so the failures were already there. None is on the known-failures list. Details are below.f583455fd. The second commit changes onlyinvalidate_group_search_cacheand the fix doc.test_group_document_collaboration.pyis the only suite that runs the real function, and it was rerun on2a38aea0awithpython -m pytest: 321 passed. Two other suites that execute realutils_cache.pyfunctions also pass there:test_content_screening_read_boundaries.py(20) andtest_ai_connection_embedding_consumers.py(8).Docs.
python functional_tests/test_docs_site_quality.pypasses 6/6.test_docs_link_integrity(2/5) andtest_docs_release_notes_integrity(0/1) match base: the generated release-notes pages are stale on V2 and were not regenerated.V2 guardrails.
run_v2_guardrails.py <worktree> 70c255508675bb0de7b7940c0ec90d5980f8b529 HEAD loginj, rerun on2a38aea0a, reports 11 changed files:[PASS] broken-access-control: 5 files[PASS] xss-sinks: 5 files[PASS] swagger-routes: 2 changed route files[PASS] python-syntax: 488 top-level files compile[PASS] malicious-pr-review: no blockers. It raised 229 Important and 14 Moderate heuristic keyword matches, mostly in the new test and fix doc: words like secret and sanitize, and the standardexec(compile(...))andimportlibharness patterns.[PASS] v2-ts-sinks: no TS/TSX files changedExisting-suite results (48 files)
test_group_document_collaboration.py(edited)test_group_workflow_fixture_parity.py(edited)test_group_document_publication.pytest_group_workflow_round_trip_preservation.pytest_orchestration_workflow_run_adapter.pytest_group_workflow_hint_seam.pytest_mcp_action_manifest_workflow.pytest_privacy_logging_telemetry_audit.pytest_chat_fact_memory_admin_placement.pytest_plugin_error_response_sanitization.pytest_mcp_outbound_logging.pytest_ai_connection_embedding_consumers.pytest_content_screening_read_boundaries.pytest_log_event_call_contract.pytest_codeql_import_cycle_lazy_imports.pytest_app_settings_import_boundaries.pytest_appinsights_logging_recursion_guard.pytest_flask_exception_console_logging.pytest_log_credential_key_redaction.pytest_workflow_alert_error_redaction.pytest_workflow_loop_inputs.pyroute_tests/test_route_blueprint_policy_inventory.pyroute_tests/test_route_unauthenticated_policy_contract.pyroute_tests/test_route_policy_test_coverage.pytest_group_directory_apis.pytest_public_settings_apis.pytest_group_file_source_apis.pytest_public_file_source_apis.pytest_group_identity_apis.pytest_group_settings_apis.pytest_public_directory_apis.pytest_public_identity_apis.pytest_group_action_apis.pytest_group_agent_apis.pytest_group_bulk_endpoint_save_writer.pytest_group_document_read_apis.pytest_public_document_publication.pytest_public_document_management.pytest_analysis_artifact_publication.pytest_workflow_cancellation.pyNameError: capture_execution_identityin the harness namespace; the 3 route and contract tests passtest_workspace_authoring_backend.py'azure.core' is not a packagetest_agents_plugins_blueprint_fix.pyapp, andconfig.pyneeds Cosmos credentialstest_plugin_tool_agent_security_audit.pyVERSION == 0.242.072test_logged_core_plugins.pytest_unified_logging_entrypoint.pytest_route_backend_models_safe_logging.pyroute_backend_models.pyreturning raw exception texttest_logging_tag_standardization.pyWorkflows/WorkflowDrafts/WorkflowAssisttags in files this PR doesn't touch. The[PLUGINS]tag used here is standard and already documented, and the tag-doc sets match base.test_agent_delegation_action.pyDocumentation
### **(v0.261.216)**section at the top ofdocs/explanation/release_notes.md; every existing section is unchanged.docs/explanation/fixes/LOG_INJECTION_CODEQL_ALERTS_FIX.md.Security checklist
@swagger_route(security=get_auth_security())sanitize_settings_for_user()sk-FAKE-log-injection-….After the V2 merges
V2 moved twice while this PR was open. Each move was merged in with
git -c rerere.enabled=false merge; nothing was rebased or force-pushed..214), merged at the user's request, asa83340d7e..215), merged at the coordinator's request, as7f90c5d7d, the current head.Conflicts. Both merges conflicted only in
config.py(VERSION) andrelease_notes.md(the top section). Each was resolved by hand, and rerere replayed nothing:VERSIONstays0.261.216, and the.216section sits directly above V2's newest section. Every other file merged without conflicts.The PR's diff hasn't changed.
git diff 86f3f3629...HEADshows 11 files, +1029/−17, with the same per-file numstat as the PR's own commits against70c255508.VERSIONline:.215now,.213originally.config.pyandrelease_notes.mdchange only by theVERSIONline and the new.216section. Every V2 release-notes section is byte-identical.functions_appinsightsimport on L23 is unchanged.Results on
7f90c5d7d, with the same venv, one process at a time:python -u functional_tests/test_log_injection_codeql_alerts_fix.py: passed, 44 tests.python -O -u functional_tests/test_log_injection_codeql_alerts_fix.py: passed, 44 tests.test_docs_site_quality.pypasses 6/6 andtest_docs_app_surface_coverage.py7/7.test_docs_link_integrity(2/5) andtest_docs_release_notes_integrity(0/1) fail exactly as on a clean86f3f3629worktree: the same 11 broken links, plus the stale generated release-notes pages. The only differences are 2 more links checked, both valid ones from the fix doc, and.216as one more release missing from the stale pages.test_workflow_result_privacy_import_cycle.py(11),test_workflow_result_chat_routes.py(12),test_workflow_result_review_paths.py(2),route_tests/test_workflow_result_context_policy.py(38) andtest_workflow_draft_instructions_assistant_gate.py(12/12).test_group_document_collaboration.py(321 passed),test_group_workflow_fixture_parity.py(60 passed),test_content_screening_read_boundaries.py(20, OK),test_ai_connection_embedding_consumers.py(8, OK) and the 3 route-policy suites (10/10, 5/5, 2/2).86f3f3629worktree gives an identical tail for each of them; the only differences are wall-clock times and the versiontest_plugin_tool_agent_security_audit.pyreports (.216here,.215there).-O, and every run failed at least one test. The 7 workflow unwraps now target L1936–L2372. The sources were restored byte-identical, andgit statuswas clean afterwards.run_v2_guardrails.py <worktree> 86f3f3629731427f13c08648478cc92ddb781651 HEAD loginjreports the same 11 changed files, and all six checks pass:[PASS] broken-access-control: 5 files[PASS] xss-sinks: 5 files[PASS] swagger-routes: 2 changed route files[PASS] python-syntax: 491 top-level files compile[PASS] malicious-pr-review: no blockers, with the same 229 Important and 14 Moderate keyword matches as before[PASS] v2-ts-sinks: no TS/TSX files changed7f90c5d7d. TheCodeQLcheck reports "No new alerts in code changed by this pull request", with 0 annotations. The merge-ref alert listing is under CodeQL.A one-off failure. On the first merge head,
a83340d7e, one normal-mode run of my test failed a single case, the[app-optimized]cold-import probe. That run took 405s, about three times as long as usual, and its output wasn't kept, so the cause isn't confirmed. Since then the probe has passed every time:a83340d7e7f90c5d7d, normal and-O[app-*]probes on7f90c5d7d, 10 of 10The probe goes through the shared
test_support/offline_bootstrap.offline_app_imports, which 25 other functional suites also use and this PR doesn't change. If it fails again, its assertion message includes the last 8000 characters of the subprocess's stdout and stderr, so the cause will be visible.Decisions made autonomously
field_count. It logs no values and no key names, throughlog_eventwith the[PLUGINS]tag the module already uses. This deliberately reduces what that log line shows: it used to print the whole request body. Key names were left out because a valid request always sends the same eight keys, plus up to two deprecated ones, and before validation they are whatever the client sent.user_idis not wrapped. It comes from the signed-in session and is not flagged. It is listed under Follow-ups.cache_keyis wrapped on the two flagged cache lines. GitHub's reported taint there comes fromdoc_scopeand the partition key, but wrapping every value on those lines costs nothing; for the hex hash it is a no-op._build_logger_extra,_normalize_extra_keyand the debugcategoryare unchanged. All 16 SARIF flows for 2551–2554 pass throughsanitize_log_messageand reachextraassc_message, so no path needed those.test_group_document_collaboration.pyandtest_group_workflow_fixture_parity.pynow receivesanitize_log_messagethrough a new helper,test_support/log_sanitizer.py, which runs it fromfunctions_appinsights.pysource. A stub would hide its real behavior.functions_authenticationbeforeroute_backend_workflows, asapp.pydoes. A coldimport route_backend_workflowson its own already fails on base withNameError: enabled_required, from an existingfunctions_settings↔functions_authenticationcycle. That cycle is listed under Follow-ups.functions_appinsightsstubs that provide onlylog_event, intest_support/app_stubs.pyand several group and public harnesses, were left alone. None of them loadsutils_cacheorroute_backend_workflows, and the suite run confirms it.group_idinstead of wrapping it inside the log call.utils_cache.py'sInvalidated {count} cache entries for group {group_id}line. That line also carries a pre-existingpy/clear-text-logging-sensitive-dataalert. Its source is_get_trusted_group_upload_scope_ids()inroute_frontend_chats.py, and the query treats the "trusted" name as sensitive.CodeQLcheck reported that existing alert as 1 new high-severity alert.group_id = sanitize_log_message(group_id)on the line before the call. The log line keeps its base text, and the alert keeps its identity. The logged text is the same either way.CodeQL
Pull request check
f583455fd. TheCodeQLcheck failed with "1 new alert including 1 high severity security vulnerability". That alert, #3936, waspy/clear-text-logging-sensitive-dataatutils_cache.py:725, and it wasn't really new: it duplicated V2's existing alert #3405 from L711. That commit had wrappedgroup_idinside the line, which changed its fingerprint (Decision 9). No newpy/log-injectionalert was reported.2a38aea0a. TheCodeQLcheck passed: "No new alerts in code changed by this pull request", with 0 annotations. This analysis also marked #3936fixedon the PR ref. All 11 checks on the head pass:Analyze (python),Analyze (javascript-typescript),Analyze (actions)broken-access-control-check,xss-sink-check,swagger-route-check,syntax-check,malicious-pr-security-review,enforce-branch-flowlicense/cla7f90c5d7d, the V2 merge head, analyzed as merge commitf75d75bec. TheCodeQLcheck passed again: "No new alerts in code changed by this pull request", with 0 annotations, and the same 11 checks all pass. While only the Actions analysis had uploaded, the check briefly showed neutral with "2 configurations not found". It passed once the Python and JS/TS analyses had uploaded.The 19 alerts on
refs/pull/1597/mergeCode-scanning reads work from this session again; for a while they returned 403, and I didn't change auth. Everything below came from the API after the third analysis.
What the merge ref shows.
alerts?ref=refs/pull/1597/mergelists one alert of any rule: #3936, which isfixed. Nopy/log-injectionalert appears there, in any state.alerts/<n>/instances?ref=refs/pull/1597/mergereturns no instances for any of the 19.fixedinstance on the merge ref.Why none of the 19 can read
fixedthere.codeql/python-queries1.8.11, the pack in V2's base analysis. It also loads thecodeql-action/pr-diff-rangeextension, which keeps only results located on changed lines.86f3f3629(1875782707) has 2469.fixedonly onpaullizer-react-v2-ui, once the branch analysis runs after the merge.What the merge-ref analysis does prove.
The locations below are V2's current ones at
86f3f3629, read from the alerts API. All 19 are open there, and V2 has no other openpy/log-injectionalert. #1592 and #1596 moved 2120–2126; the other 12 are where they were at70c255508. The merge head's changes sit on these same lines.functions_appinsights.py:419.replacechain after the regex insanitize_log_messagefunctions_appinsights.py:515functions_appinsights.py:531functions_appinsights.py:553utils_cache.py:215utils_cache.py:265utils_cache.py:501utils_cache.py:571utils_cache.py:635utils_cache.py:711group_idrebound before the callutils_cache.py:776route_backend_workflows.py:1936route_backend_workflows.py:1937route_backend_workflows.py:1944route_backend_workflows.py:1945route_backend_workflows.py:2358route_backend_workflows.py:2365route_backend_workflows.py:2372route_backend_plugins.py:2214Why the last column should hold:
codeql/python-queries1.8.11, the same as the merge-ref analysis and V2's base analysis, and reports none of the 19 on this branch (see Testing / validation).Other alerts
fixedthere. Its V2 original, #3405 atutils_cache.py:711, is still open on V2, with its fingerprint unchanged.Follow-ups
utils_cache.py: the personal fingerprint error (user_id), the cache-expired and cache-miss debug lines (cache_key), the cache read and write error lines, and the_debug_printcalls.route_backend_workflows.py: the workflow cancellationuser_id.route_backend_plugins.py:logging.info("Validated plugin settings: %s", updates), which logs only the validated booleans.import route_backend_workflowsfails withNameError: enabled_required, throughfunctions_settings↔functions_authentication.utils_cache.py: 2 results. One is the group-invalidation log line (Decision 9), the other the_debug_printstdout line. Both trace back to_get_trusted_group_upload_scope_ids()inroute_frontend_chats.py, so the logged value is a group ID that the query flags because of the "trusted" name. They look like false positives and are worth triaging on their own.functions_appinsights.py: 9 results in its logging calls, at base L375–L553. Their sources are Key Vault secret names and references and file-sync strings. This PR didn't analyze them.70c255508and on a clean V2 tip,86f3f3629:test_workflow_cancellation.py(3 runner tests)test_workspace_authoring_backend.py(1)test_agents_plugins_blueprint_fix.pytest_plugin_tool_agent_security_audit.pytest_logged_core_plugins.pytest_unified_logging_entrypoint.pytest_route_backend_models_safe_logging.pytest_logging_tag_standardization.pytest_agent_delegation_action.py(14)