docs: development guide and AGENTS.md

AGENTS.md

@@ -0,0 +1,32 @@
+# Working in this repository
+
+Two Python packages: `autodev/` (GitHub issue → PR by Claude Code) and `mia_data/` (dataset → published crops),
+both Temporal workflows with activities that talk to GitHub and the LSF cluster via csub. `mia_data` imports from
+`autodev`, never the reverse. Full guide: [`docs/development.md`](docs/development.md).
+
+## Before you push
+
+```
+pixi run fmt        # format + safe autofixes
+pixi run lint       # ruff check + format check
+pixi run typecheck  # mypy
+pixi run test       # pytest, incl. Temporal replay tests; needs no GitHub, LSF or Temporal server
+```
+
+CI runs lint, typecheck and test. Keep them clean; do not add `type: ignore` or `noqa` without a reason.
+
+## Rules that will bite you
+
+- **Activity result dataclasses in `models.py` are wire format.** They live in recorded Temporal histories. Add
+  fields with defaults; never rename or remove one. (`VerifyResult.n` keeps its name for this reason.)
+- **Workflow code is deterministic.** If `test_replay_*` fails, your change alters the commands a workflow issues
+  and will break in-flight runs. Guard with `workflow.patched()` and re-record the fixture (see the guide).
+- **No real paths or hostnames in tracked files.** `git diff | grep -n 'rokickik\|/nrs/'` must be empty.
+- **Output of sandboxed jobs is untrusted.** Read it with `read_untrusted`, reduce it to a small typed result,
+  `neutralize` it before posting to GitHub.
+
+## Style
+
+Descriptive names (no single-letter attributes), a plain-language line on each module, class and non-obvious
+method, type hints on object parameters, line length 120. Mark deliberate shortcuts with a `# ponytail:` comment
+naming the ceiling and the upgrade path.

README.md

@@ -72,7 +72,7 @@ Agents are untrusted code driven by text from issues, so they never run on the h
 |---|---|
 | `autodev/` | autodev: workflows, activities, CLI; GitHub client and csub job runner shared with mia-data |
 | `mia_data/` | mia-data: workflows, activities, harness job specs, CLI (`pixi run md`) |
-| `docs/` | setup guides and the [mia-data workflow overview](docs/mia-data-workflow.md) |
+| `docs/` | setup guides, the [mia-data workflow overview](docs/mia-data-workflow.md) and the [development guide](docs/development.md) |
 | `notes/` | design notes, runbook, specs |
 | `scripts/` | `autodev-up.sh`, `mia-data-up.sh` |
 | `tests/` | unit tests (`pixi run test`); no GitHub, LSF or Temporal server needed. `pixi run lint` runs ruff, `pixi run typecheck` runs mypy |

docs/development.md

@@ -0,0 +1,89 @@
+# Developing in this repository
+
+How to change the code here without breaking what is running. The README explains what autodev and mia-data do;
+this page is about working on them. Agents start from [`AGENTS.md`](../AGENTS.md), which is the short version.
+
+## Setup
+
+Everything runs through [pixi](https://pixi.sh). `pixi install` creates the environment (Python 3.12, Temporal
+SDK, httpx, pydantic, csub, and the test and lint tools). No GitHub token, LSF or Temporal server is needed for the
+checks below; they are only needed to run a deployment ([`docs/autodev-local-setup.md`](autodev-local-setup.md)).
+
+## The four checks
+
+Run all of these before pushing. CI runs the last three in this order on every pull request and push to main.
+
+| command | what it does |
+|---|---|
+| `pixi run fmt` | formats with ruff and applies its safe autofixes |
+| `pixi run lint` | `ruff check` plus a formatting check; fails on drift |
+| `pixi run typecheck` | mypy over `autodev/` and `mia_data/` with `check_untyped_defs` |
+| `pixi run test` | pytest over `tests/`, including the Temporal replay tests |
+
+Configuration for all four is in `pyproject.toml`. Keep them clean rather than suppressing: there are no
+`type: ignore` or `noqa` comments in the source today, and a new one needs a reason next to it.
+
+## Layout
+
+| path | |
+|---|---|
+| `autodev/` | the issue → PR loop. Also home to the pieces mia-data reuses: the GitHub client (`github.py`), auth (`auth.py`), the csub job runner (`jobs.py`), git operations (`gitops.py`), the Temporal connection and logging (`cli.py`), and the up/down/ps/logs commands (`ops.py`) |
+| `mia_data/` | the dataset → crops loop. Imports from `autodev`, never the other way round |
+| `tests/` | one `test_<package>_<module>.py` per module; `autodev_helpers.py` and `mia_helpers.py` build configs and fake repos; `fixtures/` holds recorded workflow histories and sample catalog data |
+| `docs/` | setup guides, the mia-data workflow overview, this page |
+| `notes/` | design: `mia-crops-spec.md` (the mia-data spec), `autodev-runbook.md` (operations and known gaps), `agentic-dev-infrastructure.md` (why), `specs/` and `plans/` (dated, historical; they describe the code as it was when written) |
+| `scripts/` | the up-scripts that start a deployment under agentic-sandbox |
+
+Within each package the split is the same: `workflows.py` (deterministic Temporal code, no I/O), `activities.py`
+(everything that touches GitHub, the filesystem or the cluster), `models.py` (the dataclasses that cross between
+them), `config.py` (pydantic models for the YAML), `cli.py`.
+
+## Rules that are not obvious from the code
+
+**Activity results are wire format.** The dataclasses in `models.py` that activities *return* (`PrSnapshot`,
+`VerifyResult`, `MatchResult`, ...) are serialized into Temporal workflow histories. Renaming or removing a field
+breaks replay of every in-flight workflow and fails the replay tests. Add fields with defaults; never rename.
+Activity *inputs* (`JobArgs`, `StageInput`, ...) are not replayed, so they can change when worker and workflow are
+deployed together. `VerifyResult.n` is the standing example of a name kept for this reason.
+
+**Workflow code must stay deterministic, and the replay tests enforce it.** `tests/fixtures/*history.json` are
+real histories recorded from earlier versions of the workflows. If a change alters the sequence of commands a
+workflow issues, `test_replay_*` fails. When the change is intentional, guard the new behaviour with
+`workflow.patched()` so old runs keep their old path, then re-record:
+
+```
+AUTODEV_RECORD_HISTORY=1 pixi run pytest tests/test_autodev_workflows.py -k recorded_history
+MIA_DATA_RECORD_HISTORY=1 pixi run pytest tests/test_mia_workflows.py -k replay
+```
+
+**Keep the workflows thin.** Decisions that only need data in hand (`decide.py`, `crop_test.judge`) are plain
+functions, tested directly. Everything with a side effect is an activity, called by name so the workflow module
+never imports the activity implementations.
+
+**No real paths or hostnames in tracked files.** Configs use placeholders; the real `autodev*.yaml` and
+`mia_data.yaml` are git-ignored. Before committing, `git diff | grep -n 'rokickik\|/nrs/'` should find nothing.
+
+**Anything a sandboxed job wrote is untrusted.** Read it through `read_untrusted` (size-limited, no symlinks),
+reduce it to a small typed result before it enters workflow history, and run it through `neutralize` before
+posting it to GitHub.
+
+**Shared helpers take a `Deployment`, not a `Config`.** autodev's auth, Temporal connection, logging and ops
+helpers are annotated with the `Deployment` protocol in `autodev/config.py`, which both packages' configs satisfy.
+If a helper needs a new config field, add it to the protocol and to both configs.
+
+## Style
+
+- Descriptive names everywhere except loop variables; no single-letter instance attributes.
+- A one-line module docstring saying what the module is for, a sentence or two per class, and a short plain-language
+  note on any method whose purpose is not clear from its name. Not full docstrings: this is not a public API.
+- Type hints on parameters that take objects (`Config`, `GitHub`, `Pins`, ...). Import them under `TYPE_CHECKING`
+  when a runtime import would create a cycle.
+- Line length 120, enforced by the formatter. Long-line errors are ignored by ruff; keep lines readable by eye.
+- A deliberate shortcut with a known ceiling gets a `# ponytail:` comment naming the ceiling and the upgrade path.
+  `grep -rn "ponytail:" autodev mia_data` lists them.
+
+## Pull requests
+
+Branch from `main`, keep one topic per PR, and let CI run. Both deployments poll GitHub, so a PR that changes
+activity behaviour should say in its description whether in-flight workflows are affected and what happens to
+them on deploy. The autodev bot opens PRs here too; review them like any other.