Repository navigation
fix(csv): build CSV artifacts from authorized action rows - #1308
Conversation
A telemetry agent retrieved 900 BatteryVoltage1 samples and was asked for a CSV. The published artifact held 3 rows. A follow-up conversation published 2 rows of server discovery metadata. Three defects combined. 1. build_generated_file_export() let any assistant-rendered table outrank the authorized action rows, so the illustrative excerpt the model pasted replaced the 900-row dataset. CSV now prefers the authorized rows whenever the assistant table is an excerpt of them, matched on shared columns and cell values. A table the model composed itself stays authoritative. 2. CSV never received the publication contract JSON and XML already state, so the model reported that it could not attach files and pasted a sample instead. build_generated_file_output_guidance() now tells the model the server serializes and attaches the artifact, and forbids pasting a row sample in its place. The one-clarification rule is unchanged. 3. A turn whose reply was the schema clarification still published a CSV, assembled from whatever incidental discovery rows the turn produced. That turn now publishes nothing and waits for the user's answer. Version: 0.260.007
…arifications Follow-up to the same customer conversations. Even with the authorized rows winning, two requests still had nothing to export. Discovery calls diluted the dataset. A turn that ran list_instances, list_parameters, then two pages of list_parameter_history merged all four into one table with a Source action column, so the artifact carried lookup rows alongside the samples. Action rows are now grouped by action, so paged calls to one action stay one dataset, and a retrieval result that dwarfs the turn's other calls is used on its own. "create a csv" referring to earlier data had no rows. Action results were read from the current turn only, so a follow-up request exported nothing even though the conversation already held the full result set in its stored citations. When a turn produces no assistant rows and no action rows, the export now reaches back through recent assistant turns, rehydrates their full citation payloads from the stored artifact records, and uses the most recent turn that gathered rows. The reach-back is skipped whenever the turn gathered its own rows, so rows are never double counted, and the conversation is reauthorized before any message read. Answering the schema clarification did nothing. The reply that answers it does not mention CSV, so the request was not recognized and no artifact was published. An unanswered clarification now carries the pending CSV request forward to exactly the next user turn, unless that turn requests a different format. The passthrough contract established by the original request carries with it. Version: 0.260.008
Update: earlier-turn reach-back and answered clarifications (
|
| Reply after the clarification | Pending format |
|---|---|
yes, one row per sample |
csv |
actually make it a pdf |
none |
| reply with no prior clarification | none |
Verified end to end
| Case | Result |
|---|---|
| Real turn with discovery calls + pasted sample | 900 rows, structured function result |
create a csv, data only in an earlier turn |
900 rows, earlier action result |
create a csv, data in the current turn |
900 rows, current turn only, no double count |
yes, one row per sample after a clarification |
900 rows published |
| Clarification turn itself | no artifact |
| Deliberate assistant table | assistant table wins |
| Two small comparable action groups | combined with Source action |
Validation
test_generated_csv_uses_authorized_action_rows.py 12/12, test_assistant_table_csv_artifact.py 36/36, test_generated_json_xml_exports.py 7/7, test_tabular_background_generated_exports.py 8/8, test_tabular_passthrough_heterogeneous_rows.py 4/4, test_tabular_row_orchestration_scale.py 15/15, route blueprint policy 6/6, route unauthenticated policy 4/4. py_compile, diagnostics, and Windows-safe git diff --check clean.
Tradeoff worth a reviewer's eye
The pending CSV request survives exactly one user turn. If the user changes topic immediately after a clarification instead of answering it, and an earlier turn holds rows, a CSV can still be published. The guards are: only one turn, cancelled by any other requested format, and cancelled by an empty reply. Happy to tighten this to an explicit affirmative or schema-shaped reply if preferred.
The reach-back read every artifact of a turn just to learn whether it held rows, which capped the window at 5 assistant messages and made a wider one expensive. It also took the first turn with any rows, so a newer memory or discovery call outranked the dataset a request actually meant. Compaction already leaves the answer on the message. A 54KB telemetry result is stored on the assistant message as a 677 byte citation that still carries row_count, the column names, and a remaining_items marker for the dropped tail. That is enough to decide which action holds the data without reading a single payload. Selection is now two phase. select_prior_turn_action_citations() scans the compact citations already returned by the message query, groups them by action, and picks the newest group whose recorded rows clear the dataset threshold, falling back to the most recent rows when no turn has a dataset. Only the winning group's artifacts are then fetched and expanded. The window now follows conversation_history_limit from admin settings rather than a fixed 5, so anything the model can still see in context is reachable by the exporter, capped at 40 messages. Selection is pure and lives with the other row-shape logic; the route keeps authorization and I/O. A guard test pins the compaction assumption so a change to the compaction limits fails loudly instead of silently degrading the scan. Version: 0.260.009
Update: scan first, read one payload (
|
| bytes | |
|---|---|
| raw action result | 54,145 |
| compacted citation on the message | 677 |
...and the compacted copy still carries row_count: 900, the column names, and a remaining_items: 895 marker for the dropped tail. That is enough to decide which action holds the dataset without reading a single artifact payload.
Two-phase selection
Phase 1 (free). select_prior_turn_action_citations() scans the compact citations already returned by the message query, groups them by action, and picks the newest group whose recorded rows clear the dataset threshold. Falls back to the most recent rows when no turn holds a dataset.
Phase 2 (one targeted read). Only the winning group's artifacts are fetched and expanded.
Measured on a three-turn window where the newest turns hold a get_facts and a list_instances call and the dataset is two turns back:
phase 1: scanned 2,131 compact bytes, 0 payload reads
m3 get_facts estimated rows = 1
m2 list_instances estimated rows = 1
m1 list_instances estimated rows = 1
m1 list_parameter_history estimated rows = 450
m1 list_parameter_history estimated rows = 450
phase 2: fetched m1_artifact_2 + m1_artifact_3 -> 900 rows recovered
This also fixes a correctness bug in the previous commit: it took the first turn with any rows, so the newer 1-row get_facts would have won over the 900-row dataset.
Window now follows admin settings
PRIOR_TURN_ACTION_RESULT_MESSAGE_LIMIT = 5 is replaced by conversation_history_limit from admin settings, capped at 40 messages. The rule is now easy to state: if the model can still see the turn in its context, the exporter can still use its data. Same principle as answering from citations instead of re-querying AI Search.
Phase 1 costs nothing extra per message, so the wider window is effectively free.
Structure
Selection is pure and lives with the other row-shape logic in functions_generated_file_exports.py; the route keeps authorization and I/O. That also makes it directly testable without Cosmos.
Guard on the load-bearing assumption
test_row_count_estimate_survives_citation_compaction builds a citation through the real build_compact_agent_citation and asserts the estimate still returns 900, both with an explicit row_count and with only the truncated list marker. If COMPACT_VALUE_MAX_LIST_ITEMS or the marker shape ever changes, this fails loudly instead of the scan silently degrading.
Validation
test_generated_csv_uses_authorized_action_rows.py 15/15, test_assistant_table_csv_artifact.py 36/36, test_generated_json_xml_exports.py 7/7, test_tabular_background_generated_exports.py 8/8, test_tabular_passthrough_heterogeneous_rows.py 4/4, route blueprint policy 6/6, route unauthenticated policy 4/4. py_compile, diagnostics, and Windows-safe git diff --check clean.
Still worth a reviewer's eye
Selection is recency plus volume, not relevance. If a conversation holds two different datasets, "create a csv" gets the most recent substantive one, not necessarily the one the sentence refers to. Matching the request against stored function_arguments and column names is possible but I did not want to add fuzzy relevance scoring without evals behind it.
50f8a12
into
microsoft:Development
1 participant
Summary
Two customer conversations asked a telemetry agent to retrieve
BatteryVoltage1and create a CSV. The agent retrieved 900 authorized samples. The published artifacts held 3 rows and 2 rows of server discovery metadata.Reproduced against the real extraction code:
...and create a csvexplicit_format_conversioncreate a csv, model asked a clarificationThe data was retrieved, authorized, and passthrough-eligible the whole time. A control run with the same question and no pasted sample already produced the correct 900-row CSV, so this was precedence and prompting, not plumbing.
Three defects
1. Assistant prose outranked the authorized rows
Any assistant-rendered table won unconditionally. The model pasted a 3-line illustrative CSV block to show the schema, and that excerpt replaced the dataset.
CSV now prefers the authorized rows when the assistant table is an excerpt of them.
_assistant_rows_sample_function_rows()requires strictly fewer rows, shared columns present in the action rows, and every assistant row's cell values to match a real action row. A table the model composed itself, such as a filtered or selected set, is not an excerpt and stays authoritative.2. CSV never stated the publication contract
JSON and XML guidance says the server will attach the file and explicitly forbids claiming otherwise. CSV returned only the clarification text, so the agent said:
...and then hand-wrote the 3-row sample that defeats defect 1. Rational behavior given what we told it.
build_generated_file_output_guidance()now adds the same contract for CSV. The one-clarification rule is unchanged.3. A clarification turn still published a CSV
In scenario B the model correctly asked one schema clarification and produced no data. The finalizer ran anyway, scraped the incidental
list_instances/list_parametersresults, added aSource actioncolumn, and shipped a CSV of server metadata. That turn now publishes nothing and waits for the user's answer.Changes
application/single_app/functions_generated_file_exports.py- excerpt-aware row precedence for CSV,CSV_PUBLICATION_GUIDANCE, clarification-reply gateapplication/single_app/config.py- version0.260.007functional_tests/test_generated_csv_uses_authorized_action_rows.py- newValidation
test_generated_csv_uses_authorized_action_rows.py(new)test_assistant_table_csv_artifact.pytest_generated_json_xml_exports.pytest_tabular_background_generated_exports.pytest_tabular_passthrough_heterogeneous_rows.pytest_tabular_row_orchestration_scale.pypy_compileclean, editor diagnostics clean, Windows-safegit diff --checkclean.Regression coverage kept explicit:
test_structured_action_results_combine_and_preserve_assistant_prioritystill passes, so a deliberate assistant table continues to outrank action rows.Known limitation
The excerpt match is exact on cell values. If a model paraphrases or rounds the rows it pastes, the excerpt is not recognized and the old behavior applies for that turn. Fix 2 is the primary defense there, since it instructs the model not to paste a sample at all.
Notes