Skip to content

Add catalog-native FlowBlind Research Teammate - #163

Merged
ryansok merged 6 commits into
microsoft:mainfrom
LizaMalinina:add-flowblind-research-teammate-v2
Oct 15, 2026
Merged

ryansok merged 6 commits into
microsoft:mainfrom
LizaMalinina:add-flowblind-research-teammate-v2

Conversation

@LizaMalinina

@LizaMalinina Liza (LizaMalinina) commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds the reusable catalog-native flowblind-research-teammate-v2@2.0.0
package under agents/flowblind-research-teammate-v2/.

The public surface contains exactly two tools and actions:

  • flowblind-prepare → prepare-study, confirmation Disabled
  • flowblind-run-verify → run-and-verify-study, confirmation Enabled

Regional agreement and vector/time readiness are the only advertised methods.
The retained hidden-flow method is supported privately, and its bytes are
intentionally bundled in both images, but no hidden action, tool, or mapping is
advertised. This is a routing and access boundary, not secrecy. Private
activation requires the exact separately approved and allowlisted overlay plus
V2-LIVE-004; public demos keep it disabled.

Type of change

  • New agent (under agents/<name>/)
  • Update to an existing agent
  • New starter kit (under starter-kits/<name>/)
  • Update to an existing starter kit
  • Schema change (docs/schemas/**)
  • Workflow / script change (.github/**)
  • Documentation only
  • Other (describe below)

Related issue / tracking

n/a

Schema impact

  • No schema changes
  • Schema changes are backward compatible
  • Schema changes are breaking — add the breaking-change label and link the rollout plan below

Validation checklist

  • Documentation (README.md, schemas) is updated to match the change.
  • No hand-edits to .auto-registry/** (it is generated post-merge).
  • No OS / editor artefacts committed (.DS_Store, .env, *.swp, .idea/, .vs/, etc.).
  • If model-weight files are added, they are Git-LFS tracked, <=5 GB each, and in an allowed format. No model-weight files are added.
  • No secrets, customer data, or internal-only URLs are committed.
  • All Markdown links resolve.

Reviewer notes (optional)

Changes from the Discovery Studio review

  • Tool names are now flowblind-prepare (17) and flowblind-run-verify (20),
    within Discovery's 24-character limit. Each tool uses one compute node named
    worker.
  • Publication writes and syncs a temporary file and renames it into place. No
    hard links are used, so Studio's /output (which rejects link() with
    EPERM) can store results. A rejected rename now reports
    output-storage-unsupported instead of concurrent-publication-conflict.
  • Tool and agent text tell the model to pass inputMounts at
    mountPath: /mnt/input for the selected data.
  • The router accepts everyday comparison phrasing ("Compare the reference and
    candidate values across spatial regions"). The v2 router and the retained
    regional runtime now share one vocabulary, so they cannot disagree.
  • Both Dockerfiles drop FROM --platform and pin the linux/amd64 base
    manifests, so az acr build --platform linux/amd64 works.
  • The README documents that Studio's agent-level approval must be enabled,
    because Studio does not apply per-tool confirmation.
  • Pearson reports variance-zero for constant fields and stays exact for very
    large values. Ordinary data keeps bit-identical values.
  • Outputs match the published schemas: recorded null optional context, the
    frame-resolution reason for wallShear, and the real vector metadata limits
    (quantity/license 256 and vectorMeaning/provenance 2048 characters)
    in the published input contract.
  • The README marks provenance and license (and the vector timeUnit and
    samplingKind) as required.
  • The private solver reports a speed envelope with any verified-unbounded
    direction as unbounded instead of an inconsistent failure.
  • Nested scenario-coverage tests are awaited (63/63).
  • Tool schemas and tags follow the stricter catalog rules merged in Community agent and starter-kit security scans and PR gates #132
    (single type, typed enums, no exclusiveMinimum, taxonomy tags).
  • Runtime, verifier, and test files use .js, and the hidden-flow Python
    requirements ship as requirements.lock.txt, so the package passes the
    source allowlist (POL-015) from Community agent and starter-kit security scans and PR gates #132. The package root package.json and
    an /app/package.json written by each Dockerfile declare
    "type": "module".

Changes from the Studio re-test

  • Studio rejects two inputMounts at the same path (Duplicate mount path detected: '/mnt/input'), so the run step now takes the dataset at
    mountPath: /mnt/input and the unchanged promoted preparation output at
    mountPath: /mnt/preparation. When nothing is mounted at
    /mnt/preparation, the run still accepts the preparation folder next to
    candidate.csv under /mnt/input, for hosts that merge attachments; a
    preparation in both places is refused. The run tool, agent text, and README
    describe the two mounts, and the run image sets
    FLOWBLIND_PREPARATION_ROOT=/mnt/preparation.
  • Prepare refusals for incomplete metadata now name each missing or empty
    required field, for example Missing required scientist metadata: provenance., and non-text vector metadata refusals name the field.
    Unexpected keys are reported without being echoed.

Changes from a local Discovery app run

  • The README's Chat model section records the tested models: GPT-5.6 Sol
    with medium reasoning followed the contract, while automatic selection on a
    Copilot Free plan chose MAI-Code-1.1-Flash, which wrote an unsupported
    conclusion without a tool result.

Publisher and submission identity

metadata.yaml intentionally retains Microsoft publisher.party: 1p,
discovery-catalog@microsoft.com, and the public Microsoft Discovery support
URL. Please explicitly confirm the 1P classification and support ownership.

The PR is submitted from the personal GitHub account LizaMalinina solely
because the Microsoft EMU account cannot create the required public fork. The
transport account does not change the proposed Microsoft publisher identity.

Public and private routing boundary

The catalog declarations expose only the two public tools/actions listed
above. The retained finite-basis hidden-flow closure is present in both images
for a separately governed private route, but it is absent from public tool
registration and agent routing. Image inspection can reveal those bytes, so
the boundary is access control and routing, not secrecy.

Execution path

Thanks for re-testing in Studio. Preparation now works there. For the run,
pass two inputMounts: the dataset folder at mountPath: /mnt/input and the
promoted prepare output at mountPath: /mnt/preparation. The run then reads
/mnt/input/candidate.csv and
/mnt/preparation/flowblind-preparation-<sha256>/. That Studio run is the
remaining live check.

Reviewed identities

  • Source head: 2a66fc1d3e4043bfd22095da9855a7eb3986897a
  • Source repository tree: 25310d35680da408b9377429c9c47fe670baa82f
  • Package tree (identical to this PR's package tree): e3575c4532fa5209757a44aaa4aabd20867f65ca
  • Package verifier SHA-256: 09c35b5745ab453c5a1c0ffded90f88cf4fb803e8830d4e5495afed52fbe37bc
  • Package lock SHA-256: 7b7a8955d8a1cb03b7cd0e983eaa788105e34bd50d730ed58d1c00b42fddfae8
  • Local review image IDs (linux/amd64): prepare
    sha256:9396b6f72dae4180642685e5d493354e05e2345ea77ac9d80e245da1d1f1105e,
    run sha256:9f94b83f69dcf510722a40210b7386b2ecc276a618c972267213a5435690e5aa

The committed tool.yaml files keep the schema-standard versioned ACR tags.

Local validation

  • .github/scripts/validate_pr.py against current main (6d2b6e3) for the
    package: passes with no findings; spellcheck clean. .github/tests: 625
    passed, both on pristine main and with the package.
  • FlowBlind package tests: 65 passed, including new tests for the separate
    preparation mount, the merged fallback, a preparation in both places, and
    refusals that name each missing required field.
  • The extended review harness from review/pr-163-validation-930555e,
    unchanged: all 22 public probes pass, including DOC-regional and
    DOC-vector; the hidden runtime cases pass, with
    PR163-HID-DIRECT-CERTIFICATE-INJECTION blocked as designed; and all 77
    schema cases validate.
  • With the dataset at /mnt/input, the prepare output at /mnt/preparation,
    and link()/linkat() blocked by seccomp (EPERM, as in Studio), the
    images complete prepare and run for WeatherBench2 HRES vs ERA5, VISC'11 PIV
    vs CFD, Purdue BT CFD vector time series, and a synthetic control. Results
    match the previously recorded values exactly. The Docker smoke test uses
    the same two-mount layout.
  • In the Windows Discovery App Preview (0.15.14) with GPT-5.6 Sol, a custom
    agent with this package's instructions ran both tools through a local test
    host that, like Studio, rejects two mounts at the same path. "Compare…"
    wording routed to regional agreement. On its first run call the agent
    mounted the data at /mnt/input and the prepare output at
    /mnt/preparation, and the run produced the recorded WeatherBench2 values.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added needs-human-review Awaiting human approval 1p-contribution Authored by Microsoft (party='1p' in metadata) pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Sep 20, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

faranm-stack added a commit to faranm-stack/discovery that referenced this pull request Sep 24, 2026
Run expanded offline checks against the unchanged microsoft#163 source SHA 7bc7e6a, verifying all 399 original package blobs.

Build both original tool Dockerfiles. Exercise synthetic public calculation controls and regression cases, actual emitted JSON schemas, metadata bounds, failure/recovery behavior, original image entry points with read-only inputs, and bounded private pinned-solver conformance cases. Save all actual inputs, outputs, logs, receipts and harness identities for 14 days. Failed or blocked required checks make the final gate fail; no assertions are weakened.

Disposable GitHub-hosted runner; 15-minute job; non-root network-disabled test containers; no secrets, Azure deployment, live private activation, model/Studio claims, upstream changes or published feedback. Existing baseline and three-line test-fix runs remain separate evidence.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@LizaMalinina

Copy link
Copy Markdown
Contributor Author

@microsoft-github-policy-service agree company="Microsoft"

Rename the tools to flowblind-prepare and flowblind-run-verify to stay within
Discovery's 24-character limit, name each compute node worker, and tell the
model to mount inputs at /mnt/input.

Publish outputs by renaming a synced temporary file instead of hard linking,
because Studio's /output rejects link() with EPERM. A rejected rename now
reports output-storage-unsupported instead of a publication conflict.

Fix Pearson for constant fields (variance-zero) and very large values, accept
recorded null optional context and frame-resolution wall-shear reasons in the
output schemas, publish the real vector metadata limits, mark provenance and
license as required in the README, accept everyday comparison phrasing in the
router, and report mixed-direction speed envelopes in the private method as
unbounded. Await the nested scenario-coverage tests.

Drop FROM --platform so ACR Tasks can build the images, document Studio's
agent-level approval, and follow the stricter catalog tool schema and tag
taxonomy.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@LizaMalinina
Liza (LizaMalinina) requested a review from a team as a code owner October 8, 2026 08:24
@github-actions github-actions Bot added 1p-contribution Authored by Microsoft (party='1p' in metadata) pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed 1p-contribution Authored by Microsoft (party='1p' in metadata) pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Oct 8, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added 1p-contribution Authored by Microsoft (party='1p' in metadata) contains-code PR includes source-code changes contains-dockerfile PR adds, modifies, renames, or removes a Dockerfile pr-validation-failed The pr-review workflow's validator failed; see inline comments. and removed 1p-contribution Authored by Microsoft (party='1p' in metadata) pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Oct 8, 2026
POL-015 rejects .mjs and .lock files under agents/. Rename the FlowBlind v2
launchers, verifier, bundled runtime, retained v1 and hidden-flow closures,
and portable tests to .js, and ship the hash-pinned Python requirements as
requirements.lock.txt (same bytes). The package root package.json already
declares "type": "module"; each Dockerfile now writes /app/package.json with
"type": "module" so Node loads /app/runtime/*.js as ES modules.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added 1p-contribution Authored by Microsoft (party='1p' in metadata) contains-code PR includes source-code changes contains-dockerfile PR adds, modifies, renames, or removes a Dockerfile pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed 1p-contribution Authored by Microsoft (party='1p' in metadata) pr-validation-failed The pr-review workflow's validator failed; see inline comments. contains-dockerfile PR adds, modifies, renames, or removes a Dockerfile contains-code PR includes source-code changes labels Oct 8, 2026
The run tool, agent instructions, and README now say that both the selected
data folder and the promoted preparation output use mountPath /mnt/input,
and that the flowblind-preparation-<sha256> folder name is never appended to
mountPath. In a local Discovery app run, a model appended it and the run tool
refused the doubly nested folder.

The README also records which chat models were tested: GPT-5.6 Sol with
medium reasoning followed the contract; automatic selection on a Copilot
Free plan chose MAI-Code-1.1-Flash, which wrote an unsupported conclusion.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added 1p-contribution Authored by Microsoft (party='1p' in metadata) contains-code PR includes source-code changes contains-dockerfile PR adds, modifies, renames, or removes a Dockerfile pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed 1p-contribution Authored by Microsoft (party='1p' in metadata) pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. contains-dockerfile PR adds, modifies, renames, or removes a Dockerfile contains-code PR includes source-code changes labels Oct 8, 2026
…ields

Discovery Studio rejects two inputMounts at the same path, so the run step
now reads the dataset from /mnt/input and the unchanged promoted preparation
output from /mnt/preparation. If nothing is mounted there, the run still
accepts the preparation next to candidate.csv under /mnt/input. The run
tool, agent instructions, and README describe the two mounts.

Prepare refusals for incomplete metadata now name each missing or empty
required field, and non-text vector metadata refusals name the field.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added 1p-contribution Authored by Microsoft (party='1p' in metadata) contains-code PR includes source-code changes contains-dockerfile PR adds, modifies, renames, or removes a Dockerfile pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed 1p-contribution Authored by Microsoft (party='1p' in metadata) pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. contains-dockerfile PR adds, modifies, renames, or removes a Dockerfile contains-code PR includes source-code changes labels Oct 14, 2026

@ryansok ryansok left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve

@ryansok
ryansok merged commit 680ab71 into microsoft:main Oct 15, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1p-contribution Authored by Microsoft (party='1p' in metadata) contains-code PR includes source-code changes contains-dockerfile PR adds, modifies, renames, or removes a Dockerfile needs-human-review Awaiting human approval pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants