TTT-Terminal-World 🐚

Give a model a command. Let it guess what the terminal will say. Wrong guess? Read the answer, learn a little, try again.

A research experiment in terminal world models: use Test-Time Training (TTT) to learn from ongoing interactions and store observations about the environment in model parameters.

The full-parameter Qwen2.5-Coder-3B SFT baseline uses configs/experiment/terminal_sft.yaml with the independent python -m terminal_world.sft_train entry point. Setting adaptive_layers: 0 removes all prime MLPs and inner updates. SFT uses ordinary backward/AdamW steps with sft_batch_size: 4 by default in this configuration: four right-padded shell samples are processed in one parallel forward/backward pass, followed by one optimizer update. Launch with torchrun --standalone --nproc_per_node=2 -m terminal_world.sft_train for two-GPU DDP: the global batch stays four (two samples per GPU), with rank-zero-only checkpointing and W&B logging. Each GPU retains a full model/optimizer replica. Use the same manifest and training exposure as TTT, then evaluate with --generate --frozen. Compare complete.exact_with_end_rate against TTT with and without --frozen; also report truncation coverage. SFT keeps real command/output history within each trajectory and clears it at trajectory boundaries (trajectory_context: true). Only the current output contributes to loss. Open/close events only reset SFT context and never consume batch slots or contribute training loss. Both training and evaluation retain the most recent complete interactions that fit after reserving the output budget. SFT evaluates without parameter updates, adding each real observation to context after prediction. The SFT learning rate is an untuned starting point.

A terminal guessing game, with a twist

Suppose the model has just seen this interaction:

Command: cat config.json
Output:  {"port": 8080}

On the next turn, all it gets is:

grep port config.json

Can it still remember that the port is 8080? If the file later changes to 3000, how much feedback does it need before it changes its mind?

The rule: each turn starts with only the current command. Past interactions can carry over only through parameter memory. The model gets neither a full filesystem snapshot nor a replay of the conversation history.

That's the question we want to explore: can a model gradually build an understanding of its terminal world through real feedback?

How does it remember?

Receive command → Predict output → Observe real output → Update memory
       ↑                                                       │
       └────────────────── Next command ───────────────────────┘

The model has two groups of parameters:

• General parameters θ: make predictions and learn, through meta-training, how to use feedback more effectively.
• Memory parameters φ (fast weights): update after each real observation, trying to retain what just happened.

Memory persists across commands, trajectories, and training windows, and can resume from a saved checkpoint. Move to a new environment, and the model brings its memory along. Whether old knowledge helps or gets in the way is part of the experiment.

Commands come from outside the model; its job is to predict outputs and learn from feedback. Data collection executes commands in real Docker containers. Offline training reads the recorded command / output pairs.

What's here so far?

• Data collection: an LLM-driven agent operates inside containers and records real terminal interactions.
• Training pipeline: a tiny model, persistent parameter memory, inner and outer training loops, and checkpoint recovery.
• Pretrained model support: standard Llama / Qwen2 imports, with an experiment configuration for Qwen2.5-Coder-3B Base.
• Evaluation and online use: continual adaptation, frozen-memory comparisons, and a Session interface for prediction and feedback.

This is a research prototype. GPU training quality, memory usage, and throughput with the actual 3B base model still need measurement. How much it remembers, how long it remembers, and whether memories get mixed up are questions for the experiments.

Give the tiny model a test run

Requires Python 3.10+. Run these commands from the project root. This uses a locally initialized tiny model that runs on CPU without downloading pretrained weights.

python -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[test]'

# Check the implementation
python -m pytest -q

# Index the existing training / test files without resampling
mkdir -p runs/tiny
python -m terminal_world.dataloader.trajectory \
  --training-files terminal_trajs_llm_train.jsonl \
  --test-files terminal_trajs_llm_test.jsonl \
  --output runs/tiny/manifest.json

# Run two training windows
OMP_NUM_THREADS=1 python -m terminal_world.train \
  --config configs/experiment/tiny.yaml \
  --manifest runs/tiny/manifest.json \
  --output runs/tiny/checkpoint.pt \
  --max-windows 2

The tiny model checks that the training pipeline works; two windows won't make it a terminal expert. For a fresh run, choose a new output directory. To continue from the saved checkpoint for up to two more windows:

OMP_NUM_THREADS=1 python -m terminal_world.train \
  --resume runs/tiny/checkpoint.pt \
  --manifest runs/tiny/manifest.json \
  --output runs/tiny/checkpoint.pt \
  --max-windows 2

Keep the manifest and source data unchanged when resuming. --epochs sets the total target epoch count; increase it if the saved run has already reached that target.

Learning rates for the Qwen experiment

configs/experiment/terminal_ttt.yaml uses an inner SGD learning rate of 0.10 with global gradient norm clipping at 1.0. The outer AdamW learning rate starts at zero, warms up linearly to 5e-4 over the first 10% of planned windows, then follows a cosine decay to 1e-5 on the final window. These are experiment settings, not tuned results.

The schedule length is fixed at the first launch: ceil(training events / unroll_events) × epochs, including each epoch's partial final window. Set --epochs to the intended total even for a short --max-windows trial. The checkpoint saves the schedule plan and completed window count, so resuming does not restart warmup. Extending beyond the original plan keeps the learning rate at 1e-5.

Scheduled runs need at least three planned windows; for very short runs, warmup is capped to leave a peak and final decay step. The tiny configuration and older checkpoints retain their constant learning rate. Both learning rates are included in the training logs.

Training in two stages

The intended experiment order is:

1. Train on terminal_trajs_llm_train.jsonl, then evaluate terminal_trajs_llm_test.jsonl.
2. Continue training on terminal_trajs_glm_medium_train.jsonl, then evaluate the GLM medium and LLM test sets separately.

Create one manifest per dataset using --training-files and --test-files. To switch stages, use python -m terminal_world.train --start-stage-from <completed-stage-checkpoint> --manifest <next-stage-manifest> --output <new-checkpoint> --device cuda --epochs 1. This preserves model weights, fast weights, AdamW state, and random state, starts the new dataset at its beginning, and restarts warmup and cosine decay for the new stage. The cumulative window count is preserved. Each stage's --epochs sets its own schedule horizon.

Use --resume for interruptions within a stage. Stage transitions require a completed source epoch and learning-rate plan, validate the source data, and reject cross-stage training/test trajectory overlap. A new stage must write to a new checkpoint path and gets a separate W&B run if tracking is enabled.

For each test, pass its manifest with python -m terminal_world.evaluate --checkpoint <checkpoint> --manifest <dataset-manifest> --device cuda --exposure-label <label>. Evaluation reads only the test split and adapts an in-memory copy. Stage two starts from the stage-one training checkpoint; test-time updates never enter subsequent training. Run the two final tests in separate invocations from the same stage-two checkpoint, and use --frozen for a comparison without adaptation.

Want to dig deeper?

What you're looking for	Start here
Model architecture and learning loops	Model code
Collecting real terminal trajectories	Collection script
Training and checkpoint recovery	Training entry point
Evaluation and online feedback	Evaluation · Session
Tiny and Qwen experiment settings	Experiment configurations
Correctness checks	Tests

For command-line options, run python -m terminal_world.train --help, python -m terminal_world.evaluate --help, or python scripts/collect_terminal_data.py --help after installing the relevant dependencies. Data collection uses requirements-collection.txt; the CUDA training environment uses requirements-training-cuda.txt.

Model code lives in terminal_world/, collection tools in scripts/, experiment configurations in configs/experiment/, and tests in tests/.

---

The terminal gives honest feedback every time. Let's see if the model takes the hint.