Repository navigation
fix(api): move seat code tiers and inbox horizon, raise volume floors - #37
Draft
crewletbot wants to merge 24 commits into
Draft
crewletbot wants to merge 24 commits into
crewletbot wants to merge 24 commits into
Conversation
Crewlet moves a seat's code access tier out of its GitHub identity (crewlet/crewlet#298): role.integrations.github.tier is gone, and the tier is role.code.access, one setting for every code host the engine drives, which a seat's GitHub App is created with and mints every token at. The engine refuses the old key as an unknown field, so a Company whose Seat set spec.integrations.github.tier would be held, and the revision it keeps running reads every tier as read_only. So the tier moves to the Seat's spec.code.access (ReadOnly, the default, Review or FullAccess), rendered as role.code.access; GitHubAccessTier is CodeAccessTier, and spec.integrations.github renders only for an app or repositories. The API refuses spec.code on a Human seat, as Crewlet does: a person acts as their own identity on every host, so a tier would cap nothing. Crewlet warns that a tier grants nothing where nothing applies it: a company with no GitHub Integration, or a seat without spec.integrations.github, since the operator never asks the engine to host code of its own. The new code-access case holds the render to that advisory, and the company-document page and the troubleshooting guide say it. The goldens, the sample Seat and the credentials guide follow, and the integrations guide points at the field. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
Crewlet's code host adds an eighth state log, CREWLET_CODE_LOG, which every node reserves at its 1 GiB floor whether or not the company hosts code (crewlet/crewlet#298). A fresh node now reserves at least 8053063680 bytes for its logs, and 10737418239 with the object store's bucket beside them, where it reserved 6979321856 and 9305762474, and it refuses to boot on a stream smaller than that. The stream is three quarters of the data volume, so the EngineClass floors on data.storage.size move from 9Gi to 10Gi with the files in S3 (a share of 7.5Gi) and from 12Gi to 14Gi with them in the stream's bucket (10.5Gi; 13Gi gives 10468982784 bytes, short of the floor). The engine harness declares a broker of 10.5Gi, three quarters of 14Gi, the e2e classes and the quickstart's class take 14Gi, the quickstart asks for about 12 GiB free for each company where it asked for 10, and the CRD suite's cases follow the floors. A company made at the old floor stops booting on the engine that adds the log: its existing logs keep their ceilings, and the new one finds no room beside them. The troubleshooting guide names that refusal and has its claim grown. An EngineClass stored below the new floor keeps its size and hands it to every Company created under it, so the installing and troubleshooting guides and DataStorage.size's doc tell a platform to raise it, and say what the API does meanwhile, which a new CRD case holds: it stores a class of each kind under lower floors and has the API server refuse a change to the spec of the one in the stream's bucket, a spec-wide rule, but spare the S3 one any change that leaves its size as it was. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298 changes what a pod's boot waits on: - The join reads every state log's trim floor once, each by its key and all at once, where it listed the fleet's floors once for the join and once more for each log. Its reads of where the fleet stands go from 500 s to 70 s: the positions' listing, 25 s; the floors, 5 s; and each of the eight logs' stats, 5 s. - The code host's log is the eighth state log, so a data pod's admission reads eight maintenance windows: 40 s becomes 45 s. - The object store's fleet record is agreed in casRetry's attempts, a bound of attempts rather than time. An attempt is lost only when someone removes the record during the boot, so the windows count the one attempt a boot otherwise meets, 85 s, where they counted 90 s. - A log's stats, and the reads that verify its checkpoint, are asked again while the broker answers none, on the boot's context, which has no deadline. Each is still counted at the 5 s term of one request. A data pod's boot is now 1206 seconds with its objects on the event stream and 1346 in S3 (was 1636 and 1776), and a seat pod's 1137 and 1187 (was 1142 and 1192). The EngineClass floors follow (a minimum data.startupSeconds of 1206, and 1346 with objects.backend S3), so a class may name a window 430 seconds shorter than before, and the largest volume whose derived window is a day is now 14199Gi. The runtime and troubleshooting pages quote the new figures and say how a state log's read is counted. The mirrors the windows are read from follow in the next commit. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298 moves sixteen declarations the boot windows are read from; fifteen are read again here against the windows of the commit before, at the head of its branch (codeHostRead), and mcpbridge.Build follows with the box URLs. Two are gone, and what decides in their place is named: - (*FleetStore).Floors: the join reads every log's trim floor once, each by its key and all at once, through (*stateLog).readFloors, readFloorsOf and (*FleetStore).Floor. trimFloor, which listed the floors once more for each log, goes from the table: only a write's fence reads it now, so no boot waits on it. - (*FleetStore).retryRaces: a fleet write is attempted again in casRetry's attempts, at most fleetCASAttempts of them, bounded by their number rather than by the client's 5 s default. The decision texts that were no longer true say what the engine does now: AgreeObjectBackend, fleetGenerations, replayable and objectsReservation. The rest moved no bound: openObjects, start, serveFleetBroker, armSteer, the queue's attach and the nats bucket's open, lookup and create keep their decisions under new digests. A log's stats and its record at a sequence used to be one request at the client's 5 s default, and are now asked again while the broker answers none, for as long as the caller's context lives, which a boot's has no end of. No mirrored declaration showed that, since the change was in the domain log's reads, so the table now names them: (*DomainLog).Stats, stateInfo and At, and jsprovision's ReadWith and hedge. The table's own doc and CONTRIBUTING's account of it say which waits the engine bounds by no time and how the windows count them. Read at this head, the join, startStateLog and bootSecrets are older than their rows, which follow crewlet#311 on Crewlet's main: the branch takes main before it merges, and they are unchanged on it. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
Crewlet now reads the addresses a pod hands its sandboxes, CREWLET_MCP_BRIDGE_URL and CREWLET_SANDBOX_OTEL_RECEIVER_URL, through a grammar of their own at every boot (sandbox.ParseBoxURL, through engine.ReadBoxURLs), and refuses to start on a value that is set and that the URL parser of a sandbox's coding agent, the WHATWG URL Standard's, cannot read (crewlet/crewlet#298): an IPv6 zone, a host holding a '%', '<', '>' or ']', one that is not UTF-8 once its escapes are decoded, or one whose last label is a number and that is no IPv4 address. The operator hands the Company's status.endpoints.publicBaseURL as both, so such an address now crash-loops the data pod, and a seat pod too while the class sends telemetry. The render judges each address the controller hands a pod's sandboxes (BootstrapInput.BoxURLs) by that grammar beside the addresses of its crewlet.yaml, and the controller holds the pods back on one Crewlet refuses, with RuntimeReady=False AddressRefused naming the variable and where the address came from. The API refuses what it can see: a publicBaseURL whose host holds a '%' (the zone among them), a '<', a '>' or a ']', or ends in a number and is not an IPv4 address written as four decimal octets, and a class's webhooks.hostname whose last label is a number. A last label a variable fills is a number only for some Companies, which the render decides per Company. The table mirrors ParseBoxURL and the helpers that judge its host, and ReadBoxURLs, against the files that copy them; mcpbridge.Build follows under a new digest, its parenthetical no longer true: the bridge's base is ParseBoxURL's answer, not Configured's test. TestContractBoxURLShapes holds the render's verdict to the engine's, which `crewlet llm doctor` reports as its boot reads it, and test/crd's TestAddressShapesFollowTheEngine holds the API's rules on a publicBaseURL to the same verdict. ErrAddressRefused now says the address may be in the pod's environment as well as its crewlet.yaml. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
Crewlet's image no longer carries Debian's git package: it builds git from the source release Crewlet pins (scripts/git-pin.env) and installs it under /usr/local (crewlet/crewlet#298). The tool servers guide said the image was Debian with git among its packages; it now says where the image's git comes from, and still that it has no Python, Node.js, uv, npm or curl, which is what a derived image is for. The same change adds a fifth node role, git, which the engine counts as unmanned only for a company that hosts its repositories in the engine (code.native), since /fleet's unmanned_roles now lists only the roles the company needs. The engine client's comments on a fleet node's roles and on the unmanned roles named the four roles alone; they now name the fifth and say that no pod the operator runs holds it. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
Crewlet's code host keeps a company's git repositories in the engine, declared under a new top-level block of the company document, code (native, repos, limits, deleted_retention_days; crewlet/crewlet#298). engineConfig passed it through, since it holds no credential and no field of this API owns it, and the engine takes it. But the host serves the repositories only from a node with the git role and a cache of them on its volume, and no pod the operator runs has either: a Company that set it would run a document declaring repositories nothing can clone or push, with /fleet listing git as an unmanned role and nothing saying so on the Company. So the API refuses an engineConfig entry named code, with a message of its own, until the operator offers the code host as a field of its own, with the pods and volumes it needs. The CRD suite holds the rule, and the overview and company-document pages say it is not offered. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
A data pod's engine serves the replicated estate to the rest of the fleet on two lanes since crewlet/crewlet#298 gave a push's proposals a lane of their own: estate.Serve registers the shared lane and the proposals' one after the other, and each registration flushes its subscription within flushBudget, 10 s. The data pod's window counted the estate service as one registration, as it was on Crewlet's main when the window was first read, so it fell 10 seconds short of the bounds it lays end to end. The second lane came in below (*Engine).serveEstate, whose own code did not change, so no mirrored digest moved with it. The same branch now registers a third lane, for its git reads, on a node with the git role. render.Bootstrap gives no data pod that role, so it moves no window, and the window says so. A data pod's boot is now 1216 seconds with its objects on the event stream and 1356 in S3 (was 1206 and 1346); a seat pod serves no estate and keeps its 1137 and 1187. The EngineClass floors follow (a minimum data.startupSeconds of 1216, and 1356 with objects.backend S3), the largest volume whose derived window is within a day is now 14197Gi, and the runtime and troubleshooting pages quote the new figures. The mirrors the count is read from follow in the next commit. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298 gives the estate service a lane for the git reads of a company's repositories, which a node with the git role answers on: serveEstate hands the service a git half on such a node, and estate.Serve registers the git lanes beside its others only then. Read at that engine (gitLaneRead), serveEstate takes its new digest, and its decision text says what it decides now: a data node registers its estate service, with a git half only where it has the git role, which no pod the operator runs has. Reading it against the windows found the count the previous commit corrects: the service registers its lanes one after the other, each one flush, and every data node registers two, the shared lane and the proposals', where the windows counted one. The proposals' lane came in below serveEstate, so its digest stayed put and nothing sent anybody to read it. The table now names what decides that count: estate's Serve, which registers each lane in turn, and lanes, which every node that serves the estate registers; and the queue's Serve and flush, which make every registration one flush within flushBudget. The same gap was open under three of the other registrations the windows count, a data pod's four and a seat pod's two. armHistory, armInboxRelay and armPresence each leave their registration to a function one call below them, eventfan's Serve, (*inboxRelay).serve and presence's (*Hub).Start, which no row read, so a second subject registered in any of them would have moved no digest. Each registers one subject at this engine, so the windows' 40 and 20 seconds stand, and the three are rows now, read at gitLaneRead. The steer answerer and the fleet broker register in their own step's code, which their rows read already. So every step that registers what a node answers is read down to the queue, as the table's own doc and CONTRIBUTING's account of it now say. codeHostRead's note said that nothing else which moved at that engine moved a bound. The change there that gave the registrations their answer counts also brought the proposals' lane, so the note now says that this one bound moved, outside the declarations it read. Nine more rows differ at this engine, none of them read here. Six moved with #298's changes to the leader read, its hedge and the fleet's positions ((*Leader).get, ReadWith, (*DomainLog).At, the nats bucket's open and lookupBucket, and (*FleetStore).Positions). Three read at crewlet#311's engine ((*Engine).bootSecrets, (*Engine).join and (*Engine).startStateLog) differ because #298's branch does not hold that change of Crewlet's main yet. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298 now bounds every read Crewlet asks again while nobody answers it (crewlet/crewlet@a5bf06830). A read asked that way from the start, which a state log's read of its state and of its record at a sequence are, is held to LookupBudget, 30 s, where its caller set no deadline. Before, it was asked for as long as its caller's context lived. A boot's context sets no deadline, so each such read on a data pod's boot now ends at 30 s: the join's read of every log's stats, eight one after the other, and the two reads that verify the last log's checkpoint past the consumers' window. A join told nothing fails the boot there, and an unread checkpoint lets it go on. The window counted each of those reads at the 5 s term of one request, because no bound existed to count. It now counts the bound the engine puts on them. The same change reads a certified listing's key index from the bucket's leader as one page of its subjects, inside the client's 5 s default. Before, it was two requests at that default: the stream's lookup, then its info filtered to the keys. So the listing of the nodes' positions on the join is 5 s shorter: 15 s for the listing, and 20 with the read-back of its one row. Every other read the windows count keeps its bound. A leader read a step makes on a context with no deadline is asked again inside the client's 5 s default, and so is the broker's limit read. So the admission, the trim floors, the object store's fleet record and the ceilings keep their counts, and a seat pod, whose boot reads no state log, keeps its window. A data pod's boot is now 1461 seconds with its objects on the event stream and 1601 in S3, where it was 1216 and 1356. That raises the least window a data pod is given, and every window derived from a volume, by 245 s, about four minutes: 25 s more for each of the ten reads of a state log, and 5 s less for the listing. A seat pod's stays 1137 and 1187. The EngineClass floors follow (a minimum data.startupSeconds of 1461, and 1601 with objects.backend S3), and the largest volume whose derived window is within a day is now 14156Gi. The runtime and troubleshooting pages quote the new figures. The troubleshooting page no longer says that a boot whose broker never answers is one only a restart ends: the engine gives such a read up and ends the boot itself, and the last line of its log names the read. The mirrors the count is read from follow in the next commit. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298 now asks a leader read again while nobody answers it, as a state log's reads already were, and puts a ceiling on every read asked that way (crewlet/crewlet@a5bf06830; #298's pushed head, crewlet/crewlet@b5dd4bb53, reads the same). Read at a5bf06830 (hedgeCeilingRead), five rows take new digests: (*Leader).get, ReadWith, (*DomainLog).At, and the nats bucket's open and lookupBucket. Their decision texts say what each decides now: - A leader read on a context with no deadline is still held to the client's DefaultTimeout, now asked again inside it while nobody answers. Before, get applied that bound to its one request itself. - A read asked that way from the start, which a state log's read of its state and of its record are, is handed its timing's Lookup, 30 s. Before, it was asked for as long as its caller's context lived. The previous commit counts a boot's reads of a state log at that ceiling. - A log's read of a record is given its crossing beside the read term only across a link, and the bucket's open gives its piece reads across a link a longer term. Neither moves a bound. The bucket's lookup is held to Lookup by ReadWith rather than by a context of its own. Get no longer decides the bound a leader read keeps. It hands its reader's term and bound to ask, ask hands them to askTimed, and askTimed asks through ReadBoundedWith, which holds a read whose caller set no deadline to the bound it is handed. The bound itself is NewLeader's, the client's DefaultTimeout. No row read any of those, so a change there would have moved no digest. They are rows now. So are the other reads the windows count that ReadBoundedWith bounds: the broker's limit read, (*Queue).budget, which is asked the same way and keeps the client's 5 s; and the timing whose Lookup bounds a state log's reads, (*Queue).provisioning and Clustered.Timing. Reading the windows' listing the same way found the count the previous commit corrects. The same change makes a certified listing's key index a listing of the bucket's subjects from its leader, one page for a list as short as the nodes' positions, where it was two requests at the client's default. keysUnder, (*Leader).Subjects, page and sorted are rows now too. LookupBudget's digest holds, and its text now names the reads it bounds as their ceiling. hedge, (*DomainLog).Stats and stateInfo did not move, and they decide what their texts say. The table's own doc, and CONTRIBUTING's account of it, now read a step's reads down to the function that bounds them, as they read its registrations down to the queue. They also take the state logs' reads out of the waits the engine bounds by no time, which leaves casRetry's agreement as the only one. Five rows differ at crewlet/crewlet@b5dd4bb53, and none of them is this change's. (*FleetStore).Positions moved after it was read, in a change that adds no request: its listing hands each row the time it was stored. (*Engine).armInboxRelay moved with the change that keeps the inbox relay's batches per domain (crewlet/crewlet@1a61828c7), and is read again where the operator follows that change: it now hands the relay the domains whose appliers announce and the check of which of the node's copies are out of service, and still makes its one registration, through (*inboxRelay).serve, whose digest holds, so no window moves. The three rows read at crewlet#311's engine differ because #298's branch does not hold that change of Crewlet's main yet: (*Engine).bootSecrets is not on it at all, and (*Engine).join and (*Engine).startStateLog differ from what that change made them. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298 now measures each node's clock against its broker's and raises clock_skew where the two part (crewlet/crewlet@cbc031701). The measure reads the instant the broker stored a node's row in the fleet's positions register, which every row of the listing now carries: the walk of the register's node class hands it on from the entry it holds already, and (*FleetStore).Positions sets it on the row. Read at that engine (clockSkewRead), Positions takes its new digest. Its decision is unchanged: the fleet's positions are one certified listing, and the measure adds no request to it, so the join's listing of the positions keeps its 20 s and no window moves. The change was made in the walk below Positions, (*FleetStore).eachPositionKey, which no row read: Positions' digest moved only because the walk's callback now takes the stored instant. A request added there, or anywhere else between Positions and the bounds the windows count for its listing, would have moved no digest at all. Read down the way the table already reads a step's reads and its registrations, the listing had two such stretches, and the functions on them are rows now: - from the walk to the certified listing it is: a fleet bucket's filtered walk, (*FleetStore).eachUnder and eachEntryUnder, and the leader reader it addresses without a request, newLeaderReader; - inside the listing, between listUnder and the functions that bound its waits: the pass's walk, stop, certify and close, and the read back of a key the pass did not deliver live, (*leaderReader).last and (*Leader).Last. Each makes no more requests than the windows count for a listing. The table's own doc, and CONTRIBUTING's account of it, now read the listing the windows count down to those functions. Four rows differ at #298's pushed head, crewlet/crewlet@b5dd4bb53, and none of them is this change's. (*Engine).armInboxRelay moved with crewlet/crewlet@1a61828c7 and is read again where the operator follows that change. The three rows read at crewlet#311's engine, (*Engine).bootSecrets, (*Engine).join and (*Engine).startStateLog, differ because #298's branch does not hold that change of Crewlet's main yet. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
Reading the positions listing's read back down to the leader read, in the previous commit, went through the bridge every read of a key's newest message in the coordination store crosses: (*leaderReader).last, then (*Leader).Last, then the (*Leader).get the windows count at the client's 5 s. The other reads of one key the windows count reach that bridge through code no row read either, so a request added there would have moved no digest: - a log's trim floor, its maintenance window and the object store's fleet record are each the store's read of a key's live value, (*FleetStore).get, then getLatest and (*leaderReader).live; - the object store's fleet record is created through (*FleetStore).create, whose createKey is a row already. Each round of that create also writes before it reads, and no row read what bounds that write either. It is a settled update, which waits out the leader's answer that another write to the key is still in flight, but no longer than the client's DefaultTimeout where its caller set no deadline: settledBucket.Update is one settledBucket.settle, which is one jsinflight Decide on the timeout settled gives the bucket, the client's own DefaultTimeout, and Decide asks the publish again only within it (internal/coord/kv/settle.go, internal/jsinflight). A longer budget in Decide, or a second request in the update, would have moved no digest, and left short the 85 s both pods count for the object store's fleet record. The objects bucket is settled where the fleet store opens it, which is not read: Crewlet's TestAConditionalWriteIsAnsweredByWhatTheLeaderDecided requires it. They are rows now, read at clockSkewRead, the engine the listing was read at, whose doc says so. Each read is one leader read of the key, as the Floor, Maintenance, createKey and AgreeObjectBackend rows already say, and each round's write one settled update at the client's 5 s, as createKey's says, so no window moves. The table's own doc, and CONTRIBUTING's account of it, now name the bridge beside the leader read's own bound, and the settled write beside the create. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298 makes a person's inbox the company's rather than the tracker's: every part of the engine that writes a person notices keeps them under one set of rules, so the horizon that ages them out is the top-level inbox.retention_days (unset is 365, refused outside 30..3650), and the tracker's own block, which held only that horizon, is gone (crewlet/crewlet@7504ffc5e). The engine refuses the old key as an unknown one, and no longer ties the horizon to the tracker a company runs. So the Company's horizon moves from spec.tracker.worklet to a block of its own, spec.inbox.retentionSeconds, rendered as inbox.retention_days. WorkletSettings and spec.tracker.worklet go, and with them the rule that admitted the block on backend Worklet alone: a company on Jira, or on no tracker, keeps a horizon like any other. The API is v1alpha1 and no release shipped the old field, so it moves with no alias. The field is declared anew, so it takes this API's form for a duration, an integer count of seconds, where the old inboxRetentionDays counted days. Crewlet keeps the horizon in whole days, so the field holds Crewlet's bounds in seconds, from a month (2592000) to ten years (315360000), a rule refuses a count that is not whole days, naming the multiple of 86400 it must be, and the render divides it by 86400. inbox is now a top-level key a field renders, so engineConfig may not set it. The render merges each engineConfig entry into the document as a top-level key, trusting the API to refuse every key a field renders, and an entry named inbox would have replaced what spec.inbox renders. Diff's typedKeys lists it too, so an edit made to it through the engine is found as drift rather than taken for a tuning block's default. A new render test, TestEngineConfigCannotSetAKeyAFieldRenders, holds the three lists to one another, each both ways: the company document's top-level keys, typedKeys, and the keys the CRD's rule refuses in engineConfig, read off the rule as the CRD states it. So a key the rule refuses that no field renders fails it, as a key a field renders that the rule lets through does. The CRD suite holds the bounds, the whole-day rule, a horizon on a company with no tracker, and inbox among engineConfig's reserved keys. TestCompanyDefaults now sends the tracker block empty to see its backend defaulted, since the block holds nothing else. The org-chart golden sets the horizon, the overview and company-document pages list inbox among the keys a Company renders, and the CRDs, the chart and the API reference are regenerated. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298 lifts a person's inbox out of the tracker (crewlet/crewlet@1a61828c7): every domain whose applier announces a person's movements keeps its notices under one set of rules, and the inbox relay a data node arms serves each of them. armInboxRelay, which a data pod's boot window counts as one registration, now hands the relay those domains (inboxSources) and the per-domain check of which of them the node's copy holds out of service ((*localEstate).outOfService), where it handed it the tracker copy's check alone, and the relay delivers the inbox's own movements, of any domain. So its digest moved, and the mirror table failed on it. Read again at the head of #298's branch (inboxLiftRead), against every file the row names: neither change is a request on the boot. The relay still serves every domain on its node's one subject, registered once through (*inboxRelay).serve, whose digest holds at that engine, and a node without data registers none. So a data pod's boot still counts the relay as one flush among its four registrations, no window moves, and the row's decision text stands. Its new digest, the same from crewlet/crewlet@1a61828c7 to that head, is recorded. At that engine 195 of the table's 198 rows match. The other three, (*Engine).bootSecrets, join and startStateLog, are recorded at crewlet#311, which #298's branch has not taken from Crewlet's main yet. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewletbot
changed the title
crewlet/crewlet#298 lets a turn hold each scratch repository of the company's code host it names, so that the host's expiry of an idle one leaves it be while the turn may still use it, and a launch hands that set to the run row it opens (crewlet/crewlet@521bd300f). (*launcher).Launch seals the set from its read until the row exists, so that a hold another call of the turn takes meanwhile waits, then has the row keep it, every later hold reaching the row before the request it is taken for, and unseals the set, or detaches it from the row, when the launch fails (launchHolds). So its digest moved, and the mirror table failed on it. Read again at the head of #298's branch (codeHoldsRead), against every file the row names: the render's code-work verdicts (internal/render/codework.go), its NoProvisioner report (internal/render/company.go) and the config reconciler's report of an empty start value (internal/controller/config_optional.go). The launch runs the same head with the same coding agent, and refuses before a box exists what newCodingRun and sandboxCredentials refuse, and a run whose environment holds a value that resolves to nothing under a key nothing declares optional, as before. The one refusal the holds add is the row's, of a set past the most one run may carry (sandbox.MaxCodeHolds), which only the repositories a turn names reach and no document foresees; and a company the operator runs hosts no code, since its engineConfig may not set code, so its turns hold nothing. No verdict, report or page moves, and the row's decision text stands. Its new digest, the same from crewlet/crewlet@521bd300f to that head, is recorded. The same change has every node answer for its running turns' holds (armCodeHolds), one more registration on each pod's boot. The windows need not count it: it is made past the recovery boundary, after /health answers 200, beside the seats' memory reads and the coding runs' live output, which they do not count either. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298's code writes name the model each was made on, and a call an agent-mode run makes through the tool bridge has no round of the engine's tool loop to take its model from. So (*agentLauncher).LaunchExecutor now resolves the run before it opens the run's session on the bridge, and the session carries the model the launch points the coding CLI at, for every bridged call to name (mcpbridge.Session.Model, crewlet/crewlet@86267fe2c). It still refuses a node with no bridge before it assembles the run's environment, now by asking whether it has one, in the words the bridge's empty endpoint was refused in (errNoBridgeURL), and it opens no session for a run executorRun refuses. The launch also hands the run row it opens the turn's code holds, sealed from their read to the row, as the run_sandbox launch does (crewlet/crewlet@521bd300f). So its digest moved, with each of the two changes, and the mirror table failed on it. Read again at the head of #298's branch (codeHoldsRead), against every file the row names: the render's executor verdict (internal/render/executor.go), its NoProvisioner report, the config reconciler's report of an empty start value, the LLMProvider API's rule on a CLI of mode Agent, the CLI agents guide and the CRD suite's reading of the engine's sign-in verdicts. An agent-mode turn on a node that serves no tool bridge is still refused, never run on the engine's own tool loop, and so is one whose CLI login cannot follow it into the remote sandbox (sandboxCredentials at phase Execute), the bridge first; a run whose environment holds a value that resolves to nothing still starts without it; and the model the session carries is the one the run is handed, so one model still serves the executor's run and every other phase, as the guide says. The holds' one refusal is reached by no company the operator runs, as at the run_sandbox launch. So the row's decision text stands, and its new digest, the same from crewlet/crewlet@86267fe2c to that head, is recorded. No row reads at standingInstructionsRead any more, so it goes, and codeHoldsRead's account carries what it said: neither the standing instructions, the run's label nor the wider task a run_sandbox brief frames is a verdict. At that engine 195 of the table's 198 rows match. The other three, (*Engine).bootSecrets, join and startStateLog, are recorded at crewlet#311, which #298's branch has not taken from Crewlet's main yet. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298 gives a coding run that names repositories on the company's own code host its credential, the base its git reaches and the values its commit hook writes, in the launch's own environment, and Crewlet now refuses a company document that sets any of those names (config.ReservedSandboxEnv: CREWLET_GIT_TOKEN, CREWLET_GIT_URL, CREWLET_GIT_TRAILER_SEAT, CREWLET_GIT_TRAILER_RUN and CREWLET_GIT_TRAILER_WORK_ITEM) in every env a run's environment is assembled from: a seat's sandbox.env, a seat's setup steps' env and providers.sandbox.setup's. The operator renders all three, so a Seat or a Company setting one was admitted and then held back by crewlet validate. The API now refuses each name at the variable's own name: a rule on SeatEnvVar.name (a Seat's spec.sandbox.env) and on the new SandboxEnvVar.name, the item type of a setup step's env, which took SecretEnvVar before. SandboxEnvVar has its shape and is its own type because an LLMProvider's env, which SecretEnvVar types, is no env a run's environment is assembled from, so the engine reserves nothing there. Each rule sits on the item's name, at constant cost, never as a list-wide rule on the step: the setup block's rules already cost 6 582 608 of the 10 000 000 a CRD's rules may. test/crd tries each rule in each of the three places, and holds them to the engine both ways (TestReservedRunVariablesFollowTheEngine): the names the engine's source reserves, the ones the rules state and names beside them are refused by the API exactly where crewlet validate refuses the document the operator renders. The credentials guide says so. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298's boot reads a third address a box dials back to the engine, CREWLET_BOX_GIT_URL (sandbox.BoxGitURLVar), the base a coding box reaches the company's code host at, through ParseBoxURL as it reads the tool bridge's and the telemetry receiver's, and refuses to start on a value that is set and unusable. ReadBoxURLs and BoxURLs.All are read again at the head of its branch against the operator's copies: the operator hands its pods the bridge's and the receiver's alone, does not offer the code host and so sets no third, and the API refuses every CREWLET_ variable a Company or a class gives the engine's pods, so no boot of its meets the third address and no verdict moves. The rows' decision texts name it and record the new digests; render's address constants say why the operator sets none. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298's (*launcher).Launch resolves the repositories a run_sandbox call names on the company's own code host, after every refusal a document foresees and before the turn's holds are sealed for the row, and hands a run given some its credential, its git base and git's configuration in the launch's own environment, never on the row (ADR-0063). Read again at the head of its branch against the operator's copies: a company the operator runs hosts no code (its engineConfig may not set code), so no call of its seats names a repository, and the head a launch runs on, its coding agent, every other refusal before a box exists and the sign-in that must reach the box are as at codeHoldsRead. The variables that environment carries are the ones the API now refuses in a Seat's spec.sandbox.env and a setup step's env. The row records the new digest, read at runReposRead. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298 makes a person's stop reach the coding runs a seat has detached, which outlive their turn and run on through a pause: until the seat is resumed the engine refuses every change such a run would make, naming who stopped the seat. That is a push its box makes to the company's code host, and, since crewlet/crewlet@0645a8eb5, a call it makes through the tool bridge of any of the seat's tools not marked read-only (mcpbridge.Session.Stopped), answered paused. Its reads still answer, and a pause without stop_running stops new work only. Seat.spec.stopRunning said only that a stop ends the turn the seat is on. Its doc, and the status field reporting it, now say what else it refuses, and that it lasts until the seat is resumed, which is why unsetting it changes nothing. The applying page's account of seat pauses says the same. The operator offers no code host, so for a company it runs the stop's reach is the bridged writes; the doc names the pushes too, as the engine's own stop_running does. Regenerated: the Seat CRD, the chart's copy and the API reference. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
crewlet/crewlet#298's (*agentLauncher).LaunchExecutor now opens its run's session on the tool bridge through bridgeSession, which sets the fields it set before and the seat's stop check, stoppedBy, the one reading the code host refuses a run's box's pushes by (crewlet/crewlet@0645a8eb5). So its digest moved, and the mirror table failed on it. Read again at the head of #298's branch (bridgeStopRead, crewlet/crewlet@ab6d97e67) against every file the row names: the render's executor verdict, its NoProvisioner report, the config reconciler's report of an empty start value, the LLMProvider API's rule on a CLI of mode Agent, the CLI agents guide and the CRD suite's reading of the engine's sign-in verdicts. The launch refuses what it refused before, in the same order: a node with no bridge, and a CLI login that cannot follow the run into the sandbox. bridgeSession always sets the check, so the bridge's new refusal of a session without one is never met. The row's decision text stands, and its new digest is recorded. What a stop refuses a run is no verdict on a launch; the previous commit says it where the operator says what a stop does. No row reads at codeHoldsRead any more, so it goes, and bridgeStopRead's account carries what it said. At that engine 191 of the table's 198 rows match. Of the other seven, three are recorded at crewlet#311, which #298's branch has not taken from Crewlet's main yet ((*Engine).bootSecrets, join and startStateLog); the other four moved with later #298 changes whose own companions read them: estate's Serve (the git write bound and the git lanes' doc), (*stateLog).start, and the embedded broker's connect and awaitLeafReady (the in-process clients and the fleet's keys). Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
A pause with stopRunning ends the turn the seat was on and, since crewlet/crewlet#298, refuses every change its detached coding runs would make until the seat is resumed: a call through the tool bridge of any of the seat's tools not marked read-only, and a push from their boxes. Seat.spec.stopRunning and status.pause.stopRunning say so, but the Paused condition's message still said only that the turn was stopped, and the Drifted and PauseNotManaged texts asked for a seat "paused, its running turn stopped". Someone reading a stopped Seat was told the engine did less than it does. The condition now adds that the seat's detached coding runs' writes are refused until it is resumed, and the asked-for state reads "paused, its running turn stopped and its runs' writes refused", as the engine's own pause_seat says both halves. A test holds both texts to the stop, and to a pause without one naming neither. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
The client's SeatPause.StopRunning and PauseSeat docs said a stop only ends the turn the seat is on. With crewlet/crewlet#298 it also refuses every change the seat's detached coding runs would make, a call of its tools that writes and a push from their boxes, until the seat is resumed, as the engine's pause_seat tool describes it. Signed-off-by: Seena Fallah <seena.fallah@ionos.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
This is the companion of crewlet/crewlet#298, which hosts a company's git repositories inside the engine. That change moves several things this operator renders, copies, runs and links. Once it merges, the operator's next run against Crewlet's
mainfails without this pull request:Land right after crewlet/crewlet#298, once
main-image.ymlhas rebuiltghcr.io/crewlet/crewlet:main. This repository's suites run that image (ADR-0002), and until then it is an engine without the code host. Between the two merges, a Company whose Seat sets a GitHub tier is held (Applied=False,EngineRefuses). It keeps running its last revision, in which the engine reads every seat's tier asread_only. Likewise, a Company that setsspec.tracker.worklet.inboxRetentionDaysis held, and the revision it keeps running carries no horizon that engine reads, so a person's notices age out at Crewlet's default of a year. So the two land back to back.What changes for someone running the operator
spec.code.access.spec.integrations.github.tieris gone. Move its value tospec.code.access:ReadOnly(the default),RevieworFullAccess. The API refusesspec.codeon a Human seat. No release has shipped the old field (the API isv1alpha1and there is no tag), so this is a plain move with no alias.spec.inbox.retentionSeconds.spec.tracker.worklet, which held onlyinboxRetentionDays, is gone: Crewlet now keeps one horizon for every notice in a person's inbox, whatever tracker the company runs, Jira or none included. Like this API's other durations, the new field counts seconds, in whole days, from 2592000 (30 days) to 315360000 (3650 days), such as 7776000 for 90 days. Unset is still Crewlet's default of a year.engineConfigmay not setinbox. This too is a plain move with no alias.data.storage.sizemust be at least 14Gi withobjects.backend: NATS(was 12Gi) and 10Gi with S3 (was 9Gi). Two things need a manual step, and the installing and troubleshooting guides say how to do each:data.startupSecondsfollows, and so does every window derived from a volume.RuntimeReady=False, reasonAddressRefused, naming the variable, rather than leaving them to crash-loop. The API refuses the cases it can see in a Company'spublicBaseURLand in a class'swebhooks.hostname.engineConfigrefusescode. This operator does not offer the engine's code host yet: no pod it runs has thegitrole or a repository cache.CREWLET_GIT_TOKEN,CREWLET_GIT_URL,CREWLET_GIT_TRAILER_SEAT,CREWLET_GIT_TRAILER_RUNandCREWLET_GIT_TRAILER_WORK_ITEMas a variable's name in a Seat'sspec.sandbox.envand in a setup step'senv(spec.sandbox.setup[].env, on a Company or a Seat). The engine refuses a company document that sets one, so such a Company or Seat was admitted and then held withEngineRefuses. Give a variable of your own, such as a token for another git host, another name. An LLMProvider'senvis not affected.spec.stopRunning, Crewlet ends the turn the seat is on and, until the seat is resumed, refuses every change its detached coding runs would make: a call through the tool bridge of any of the seat's tools not marked read-only, and a push from their boxes. Their reads still answer. The field's doc,status.pause.stopRunning, thePausedcondition's message and theDriftedandPauseNotManagedtexts now say so. Nothing in the API changes shape.What each commit follows in the engine
refactor(api): move a seat's code tier to spec.code.accessrole.integrations.github.tierand refuses it as an unknown field. The tier is nowrole.code.access(read_only,revieworfull_access; unset is read only), androle.codeis refused on a human seat. A new advisory,code.access <tier> grants nothing, is given when nothing applies the tier: the company has no enabled GitHub integration, or the seat has nointegrations.githubblock.spec.code.access, rendered asrole.code.access.GitHubAccessTierbecomesCodeAccessTier,spec.integrations.githubis rendered only forreposorapp, andcodejoins the Human seat's CEL list of agent-only fields. A newcode-accessgolden case holds the render to the advisory. The goldens, the sample Seat, and the credentials, integrations, company-document and troubleshooting pages follow.fix(api): raise the data volume floor to the engine's eight logsfix(controller): shorten the boot windows to the engine's new boundscasRetryattempts, a bound of attempts rather than time; the windows count the one attempt a boot meets unless someone removes the record (90 s becomes 85 s).dataPodBootSecondsandseatPodBootSeconds, the EngineClass's minimumdata.startupSeconds, and the runtime and troubleshooting pages follow.test(render): read again the boot bounds the engine moved(*FleetStore).FloorsandretryRaces.(*FleetStore).Floor,readFloorsandreadFloorsOftake the place ofFloors, andcasRetryandfleetCASAttemptsthe place ofretryRaces.trimFloorleaves the table, since only a write's fence reads it now.ReadWith,hedgeand(*DomainLog).Stats,stateInfoandAtjoin it, because a log's reads on the boot path are now asked again while the broker answers none. The decision texts that stopped being true are rewritten.fix(render): hold a pod back on a public address no sandbox can dialCREWLET_MCP_BRIDGE_URLandCREWLET_SANDBOX_OTEL_RECEIVER_URLare read throughsandbox.ParseBoxURL(viaengine.ReadBoxURLs). The boot is refused on a value a sandbox's WHATWG URL parser cannot read: an IPv6 zone, a%,<,>or]in the host, or a numeric last label on a host that is not an IPv4 address.publicBaseURLandwebhooks.hostnamefollow. The table mirrorsParseBoxURL, its host helpers andReadBoxURLs.TestContractBoxURLShapesholds the render's verdict to whatcrewlet llm doctorreports.docs: follow the engine image's own git and the role it adds/usr/local, with no Debiangitpackage.node.rolesgainsgit, which/fleetcounts as unmanned only for a company withcode.native.fix(api): refuse code in engineConfig until the code host is offeredcodeblock declares the company's repositories, which only a node with thegitrole and a repository cache serves.engineConfigentry namedcode, with its own message. The overview and company-document pages list it as not offered.fix(controller): count the estate service's two lanes in the boot windowflushBudget, 10 s. A node with thegitrole also registers lanes for the git reads of its repositories.main, so it fell 10 s short of the bounds it adds up. It now counts two. The git lanes add nothing, because the render gives no data pod thegitrole. A data pod's window becomes 1216 s, or 1356 s with S3 (1206 s and 1346 s after commit 3). A seat pod serves no estate and keeps 1137 s and 1187 s. The EngineClass's minimumdata.startupSeconds, the largest volume whose derived window stays within a day (now 14197Gi), thetest/crdcases, and the runtime and troubleshooting pages follow.test(render): read serveEstate again after the git laneEngine:
(*Engine).serveEstatehands the estate service a git half on a node with thegitrole, andestate.Serveregisters the git lanes only when it has one.Here:
serveEstateis read again at that engine (crewlet/crewlet@adba06740) and its new digest recorded. Its decision text now says that it hands the service a git half only where the node has thegitrole, which no pod the operator runs has. The table now also reads how many registrations a boot step makes, down to the queue:estate'sServeandlanes;(*Queue).Serveandflush, which make each registration one flush;eventfan'sServe,(*inboxRelay).serveand presence's(*Hub).Start, one call below the steps that arm the history answerer, the inbox relay and the presence answerer. Each registers one subject today, so the windows those steps take stand.The proposals' lane came in below
serveEstate, whose own code did not change, so no digest moved for it. A registration added below a step now moves one.CONTRIBUTING.mdsays so where it explains the table.fix(controller): count a state log's boot reads at their new ceilingLookupBudget, 30 s, where its caller set no deadline. Before, it was asked for as long as its caller's context lived, and a boot's context has no deadline. A leader read on such a context keeps the client's 5 sDefaultTimeout. A certified listing's key index is now one page of the bucket's subjects from its leader, where it was two requests at that default.data.startupSecondsand its S3 rule, the largest volume whose derived window stays within a day (now 14156Gi), thetest/crdcases, and the runtime and troubleshooting pages follow. The troubleshooting guide no longer says that only a restart ends a boot whose broker never answers.test(render): read the leader read and its bound again after the hedgeEngine: a leader read is now asked again while nobody answers it, as a state log's reads already were.
(*Leader).getno longer applies the client's bound to its one request itself: it hands its reader's term and bound toaskandaskTimed, which ask throughReadBoundedWith. The NATS bucket's lookup is held toLookupbyReadWithrather than by a context of its own, its open gives a bucket's piece reads across a link a longer term, and a log's read of a record across a link is given its crossing beside the read term. None of these three moves a bound.Here: five rows are read again at crewlet/crewlet@a5bf06830 and their new digests recorded:
(*Leader).get,ReadWith,(*DomainLog).At, and the NATS bucket'sopenandlookupBucket. Their decision texts say what each decides now. Eleven rows join the table, so that every read the windows count is read down to the function that bounds it:NewLeader,ask,askTimedandReadBoundedWith, which hold a leader read to the client's 5 s;(*Queue).budget, the broker's limit read, which is asked the same way and keeps 5 s;(*Queue).provisioningandClustered.Timing, whoseLookupbounds a state log's reads at 30 s;keysUnder,(*Leader).Subjects,pageandsorted, which make the listing's key index one page.LookupBudget's digest holds, and its text now names the reads it bounds. The table's own doc andCONTRIBUTING.md's account of it now read a step's reads down to the function that bounds them, and name the agreement of the object store's fleet record (casRetry) as the only wait the engine bounds by no time.test(render): read the positions listing again after the clock measureEngine: each node's clock is now measured against its broker's, and
clock_skewis raised where the two part (crewlet/crewlet@cbc031701). The measure reads the instant the broker stored a node's row in the fleet's positions register, which every row of the listing now carries:(*FleetStore).eachPositionKey, the walk of one class of the register, hands it on from the entry it already holds, and(*FleetStore).Positionssets it on the row. The listing makes no request it did not make before.Here:
(*FleetStore).Positionsis read again at that engine and its new digest recorded. Its decision is unchanged, one certified listing, so the join's listing of the positions keeps its 20 s and no window moves. The change itself was made in the walk belowPositions, which no row read:Positions' digest moved only because the walk's callback now takes the stored instant, and a request added there would have moved no digest at all. Ten rows join the table, so that the listing the windows count is read down to the functions that bound its waits:eachPositionKey, the fleet bucket's filtered walk ((*FleetStore).eachUnderandeachEntryUnder), andnewLeaderReader, which addresses the leader without a request;walk,stop,certifyandclose, and the read back of a key the pass did not deliver live,(*leaderReader).lastand(*Leader).Last.The table's own doc and
CONTRIBUTING.md's account of it now read the listing down to those functions.test(render): read the fleet store's key reads and create to the boundsEngine: the clock measure moved nothing on this path. The windows' other reads of one key (a log's trim floor, its maintenance window and the object store's fleet record) cross the same bridge to the leader read as the listing's read back. Each round of the fleet record's create also writes before it reads: a settled update, which waits out the leader's answer that another write to the key is still in flight, but no longer than the client's 5 s
DefaultTimeoutwhere its caller set no deadline.Here: eight rows join the table, read at the same engine:
(*FleetStore).get,getLatestand(*leaderReader).live, the store's read of a key's live value, which is one leader read at the client's 5 s;(*FleetStore).create, which is onecreateKey(a row already) on the store's client;settled,settledBucket.Update,settledBucket.settleandinternal/jsinflight'sDecide, which hold each round's write to that 5 s.Before them, a longer budget in
Decide, or a second request in the update or the read, would have moved no digest, and left short the 85 s both pods count for the object store's fleet record. No window moves. That the objects bucket is settled is not read here, since Crewlet's ownTestAConditionalWriteIsAnsweredByWhatTheLeaderDecidedrequires it. The table's own doc andCONTRIBUTING.mdnow name the leader read's bridge, and the settled write beside the create.feat(render): render the inbox horizon at the company's top levelinbox.retention_days(unset is 365, refused outside 30..3650), and the tracker's block, which held only that horizon, is gone. The engine refuses the old key as an unknown one, and no longer ties the horizon to the tracker a company runs.spec.tracker.worklet.inboxRetentionDaysto a block of its own,spec.inbox.retentionSeconds, rendered asinbox.retention_days. The field is declared anew, so it counts seconds as this API's durations do: a rule refuses a count that is not whole days, and the render divides it by 86400.WorkletSettings,spec.tracker.workletand the rule that admitted it on backendWorkletalone go, so a company on Jira, or on no tracker, keeps a horizon like any other.inboxjoins the keysengineConfigmay not set, and Diff'stypedKeys, so an edit made to it through the engine is found as drift. A new render test,TestEngineConfigCannotSetAKeyAFieldRenders, holds three lists to one another, each both ways: the company document's top-level keys,typedKeys, and the keys the CRD's rule refuses inengineConfig, read off the rule as the CRD states it. Thetest/crdcases, the org-chart golden, the overview and company-document pages, the CRDs, the chart and the API reference follow.test(render): re-digest the engine's inbox relay registration(*Engine).armInboxRelayhands it those domains (inboxSources) and the per-domain check of which of them the node's copy holds out of service ((*localEstate).outOfService), where it handed it the tracker copy's check alone, and the relay delivers the inbox's own movements, of any domain.armInboxRelayis read again at the engine change's branch (crewlet/crewlet@66e8ad593), against every file its row names, and its new digest recorded; the digest is the same from crewlet/crewlet@1a61828c7 on. Neither change is a request on the boot: the relay still serves every domain on its node's one subject, registered once through(*inboxRelay).serve, whose digest holds, and a node withoutdataregisters none. So a data pod's boot still counts the relay as one flush, no window moves, and the row's decision text stands.test(render): read the run_sandbox launch again after its code holds(*launcher).Launch, the run_sandbox launch, hands that set to the run row it opens. It seals the set from its read until the row exists, so that a hold another call of the turn takes meanwhile waits; then the row keeps it, and every later hold reaches the row before the request it is taken for. When the launch fails, it unseals the set or detaches it from the row (launchHolds). The one refusal the holds add is a set past the most one run may carry (sandbox.MaxCodeHolds, 1024). Every node also answers for its running turns' holds (armCodeHolds), one more registration on each pod's boot.Launchis read again at the engine change's pushed head (crewlet/crewlet@4ee3f06a5), against every file its row names, and its new digest recorded; the digest is the same from crewlet/crewlet@521bd300f on. The launch still refuses, before a box exists, what it refused before. Only the repositories a turn names reach the new refusal, and a company the operator runs hosts no code, since itsengineConfigmay not setcode(commit 7), so its turns hold nothing. No verdict, report or page moves, and the row's decision text stands. The windows need not countarmCodeHolds: it registers past the recovery boundary, after/healthanswers 200, beside the seats' memory reads and the coding runs' live output, which they do not count either.test(render): read the agent-mode launch again after the bridge's model(*agentLauncher).LaunchExecutorresolves the run before it opens the run's session on the bridge, and the session carries the model the launch points the coding CLI at (mcpbridge.Session.Model, crewlet/crewlet@86267fe2c). It still refuses a node with no bridge before it assembles the run's environment, now by asking whether it has one, in the words the bridge's empty endpoint was refused in (errNoBridgeURL), and it opens no session for a runexecutorRunrefuses. It also seals the turn's code holds to the run row it opens, as the run_sandbox launch does (crewlet/crewlet@521bd300f), so its digest moved with each of the two changes.LaunchExecutoris read again at the same head, against every file its row names: the render's executor verdict, itsNoProvisionerreport, the config reconciler's report of an empty start value, the LLMProvider API's rule on a CLI of modeAgent, the CLI agents guide and the CRD suite's reading of the engine's sign-in verdicts. Its new digest, the same from crewlet/crewlet@86267fe2c on, is recorded. An agent-mode turn on a node with no tool bridge is still refused, never run on the engine's own tool loop, and so is one whose CLI login cannot follow it into the remote sandbox. A value of the run's environment that resolves to nothing still leaves the run to start without it. The session's model is the one the run is handed, so one model still serves the executor's run and every other phase, as the guide says. The row's decision text stands. Both launch rows now record the engine they were read at ascodeHoldsRead;standingInstructionsRead, which no row reads any more, goes, andcodeHoldsRead's account carries what it said.fix(api): refuse the variables Crewlet sets in a coding run itselfconfig.ReservedSandboxEnv:CREWLET_GIT_TOKEN,CREWLET_GIT_URL,CREWLET_GIT_TRAILER_SEAT,CREWLET_GIT_TRAILER_RUNandCREWLET_GIT_TRAILER_WORK_ITEM) in every env a run's environment is assembled from: a seat'ssandbox.env, its setup steps'env, andproviders.sandbox.setup's.SeatEnvVar.name(a Seat'sspec.sandbox.env) and on the newSandboxEnvVar.name, the item type of a setup step'senv, which tookSecretEnvVarbefore.SandboxEnvVarhasSecretEnvVar's shape and is a type of its own because an LLMProvider'senv, whichSecretEnvVartypes, is no env a run's environment is assembled from, so the engine reserves nothing there. Each rule sits on the item's name at constant cost (17 per name), never as a list-wide rule on the step: the setup block's rules already cost 6 582 608 of the 10 000 000 a CRD's rules may.test/crdtries each rule in each of the three places (TestReservedRunVariablesValidation: 15 refusals and 21 admissions), andTestReservedRunVariablesFollowTheEngineholds them to the engine both ways: the names the engine's source reserves, the names the rules state, and names beside them are refused by the API exactly wherecrewlet validaterefuses the document the operator renders. The credentials guide, the CRDs, the chart and the API reference follow.test(render): read the box address reads again after the code host'sCREWLET_BOX_GIT_URL(sandbox.BoxGitURLVar), the base a coding box reaches the company's code host at. It reads it throughParseBoxURL, as it reads the tool bridge's and the telemetry receiver's, and refuses to start on a value that is set and unusable.ReadBoxURLsandBoxURLs.Allare read again at the engine change's pushed head (crewlet/crewlet@2887e830c), against every file their rows name, and their new digests recorded. The operator hands its pods the bridge's and the receiver's addresses alone. It does not offer the code host, so it sets no third, and the API refuses everyCREWLET_variable a Company or a class gives the engine's pods. So no boot of its meets the third address, and no verdict moves. The rows' decision texts name it, and render's address constants say why the operator sets none. On review, the harness'sBoxURLVerdictsdoc, which still said the engine reads two addresses, now names the three, and the rows' list of operator copies (boxURLCopies) now names the harness, so the next re-read reaches it.test(render): read the run_sandbox launch again after its repositories(*launcher).Launchresolves the repositories a run_sandbox call names on the company's own code host after every refusal a document foresees and before the turn's holds are sealed for the row, and hands a run given some its credential, its git base and git's configuration in the launch's own environment (ADR-0063).Launchis read again at the same head, against every file its row names, and its new digest recorded, read atrunReposRead. A company the operator runs hosts no code (itsengineConfigmay not setcode, commit 7), so no call of its seats names a repository. The head a launch runs on, its coding agent, every other refusal before a box exists, and the sign-in that must reach the box are as atcodeHoldsRead. The variables that environment carries are the ones commit 18 refuses. No verdict, report or page moves.docs(api): say a stop also refuses a detached run's writesmcpbridge.Session.Stopped, crewlet/crewlet@0645a8eb5), answered paused. Its reads still answer, and a pause withoutstop_runningstops new work only.Seat.spec.stopRunningsaid only that a stop ends the turn the seat is on. Its doc and the status field reporting it now say what else it refuses, and that this lasts until the seat is resumed, which is why unsetting the field changes nothing. The applying page's account of seat pauses says the same. The operator offers no code host, so for a company it runs the stop reaches the bridged writes; the doc names the pushes too, as the engine's ownstop_runningdoes. The Seat CRD, the chart's copy and the API reference are regenerated.test(render): read the agent-mode launch again after the bridge's stop(*agentLauncher).LaunchExecutornow opens its run's session on the tool bridge throughbridgeSession, which sets the fields it set before plus the seat's stop check,stoppedBy, the same check the code host refuses a run's box's pushes by (crewlet/crewlet@0645a8eb5). So its digest moved.bridgeStopRead), against every file it names: the render's executor verdict, itsNoProvisionerreport, the config reconciler's report of an empty start value, the LLMProvider API's rule on a CLI of modeAgent, the CLI agents guide and the CRD suite's reading of the engine's sign-in verdicts. The launch refuses what it refused before, in the same order: a node with no bridge, and a CLI login that cannot follow the run into the sandbox.bridgeSessionalways sets the check, so the bridge's new refusal of a session without one is never met. The row's decision text stands and its new digest is recorded. No row reads atcodeHoldsReadany more, so it goes, andbridgeStopRead's account carries what it said.fix(controller): say a stopped seat's runs' writes are refusedpause_seatsays both halves: the turn ends, and the detached runs' writes are refused.Pausedcondition's message said only that the turn was stopped, and theDriftedandPauseNotManagedtexts asked for a seat "paused, its running turn stopped". The condition now adds that the seat's detached coding runs' writes are refused until it is resumed, and the asked-for state reads "paused, its running turn stopped and its runs' writes refused".TestAStoppedSeatSaysItsRunsWritesAreRefusedholds both texts to the stop, and holds a pause without one to naming neither.docs(engine): say a seat's stop also refuses its runs' writesSeatPause.StopRunningandPauseSeatdocs said a stop only ends the turn the seat is on. They now name the refused writes too: a call of the seat's tools that writes, and a push from its runs' boxes.How it was checked
What ran, in a container with Go 1.27.2.
At b4cc2c1, for commits 1 to 7:
go test -race -count=1over the packages that need no envtest assets, cluster or engine image:internal/render: the goldens,TestStreamFloorFiguresAreTheEngines,TestTheDocsStateTheBases,TestBootstrapRefusesABoxURLNoSandboxCanDial,TestAMirroredDeclarationThatMovedFailsandTestEveryFindingHasABasis;internal/controller, without the envtest suiteTestControllers: the startup-window tests among them;internal/engineand its harness,internal/secretsync,internal/secretref,test/docs,test/enginecommitandtest/envtestbin.Every test in those packages that failed did so only because the engine binary was absent. A script checked each failing test's output for that.
Static checks: gofmt;
go vetin both modules, and with-tags e2eover./test/e2e/;go build ./...;make manifests generate chart api-docs, which leaves the tree unchanged; and the sign-off gate.A mutation check: with the box grammar's refusal of a numeric last label turned off,
TestBootstrapRefusesABoxURLNoSandboxCanDialfails.A throwaway test, not committed, that ran the new
engineConfigrule through the API server's CEL validator against the generated CRD.codeis refused with its message, and nearby names are admitted.A digest scan of all 162 mirror rows with
enginetest.Fingerprint. Every row matches the engine: 159 at the engine change's head (15c2f1a96), and the other three at Crewlet'smain(fix(engine): read secrets and build the company after a node recovers crewlet#311).At 1306e5c, after commits 8 and 9:
go test -race -count=1overinternal/controllerwithout the envtest suiteTestControllers, and overinternal/renderandtest/docs. In those packages 834 tests and subtests pass, and every one that fails does so only because the engine binary is absent: a script checked each failing test's output for that. Among those that pass:TestDataPodStartupSecondsandTestStartupFailureThreshold;TestStartupSecondsFloorIsTheBootWindow, which holds the CRD'sdata.startupSecondsfloors to the derived window;TestStartupWindowFiguresAreTheControllers, which holds every page and comment that quotes a window to it;TestAMirroredDeclarationThatMovedFails, and the docs gate'sTestRelativeLinksResolveandTestEveryPageIsInTheIndex.go vetoverinternal/controller,internal/renderandtest/crd, and over theapimodule; andmake verify-generated, which leaves the tree unchanged.enginetest.Fingerprint, over the engine's git objects:At 35ed36e, after commits 10 and 11 (the tests ran on its tree; the two commit messages were reworded afterwards, which left the tree as it was):
go test -race -count=1over:internal/controller, without the envtest suiteTestControllersandTestExecutorFollowsTheEngineSource, which reads the engine's source. Among the passes:TestDataPodStartupSeconds,TestSeatPodStartupSeconds,TestStartupFailureThreshold,TestStartupSecondsFloorIsTheBootWindowandTestStartupWindowFiguresAreTheControllers;internal/render: 591 tests and subtests pass, and each of the 531 that fail does so only because the engine binary is absent (a script checked each failing test's output for that),TestMirroredEngineDeclarationsAreUnchangedand the contract tests among them. Among the passes:TestAMirroredDeclarationThatMovedFails,TestStreamFloorFiguresAreTheEngines,TestTheDocsStateTheBases,TestCompanyGolden,TestBootstrapGoldenandTestEveryFindingHasABasis;test/docs, withoutTestEngineLinksResolve, which needs the engine binary. The one engine heading it would check, "The embedded broker's own logs" in Crewlet's deployment guide, exists at the engine change's pushed head and on Crewlet'smain;internal/secretref,test/enginecommitandtest/envtestbin.After the rewording, the window tests and
TestAMirroredDeclarationThatMovedFailsran again and pass.A mutation check: with the old terms restored in
dataPodBootSeconds,TestDataPodStartupSecondsfails.Static checks:
go vetoverinternal/controller,internal/render,test/crdandtest/docs, and over theapimodule, which compiles the edited envtest cases without running them;go build ./...in both modules; gofmt over the changed Go files;make manifests generate chart api-docs, which leaves the tree unchanged; and the sign-off gate over all eleven commits.A digest scan of all 180 mirror rows with the table's own check, over the engine's git objects:
At ab3fcc8, after commits 12 and 13:
go test -race -count=1over:internal/render, at ab3fcc8:TestAMirroredDeclarationThatMovedFails,TestCompanyGolden,TestBootstrapGolden,TestEveryFindingHasABasis,TestTheDocsStateTheBasesandTestStreamFloorFiguresAreTheEnginespass. The whole package ran on commit 13's first version, before its last four rows were added on review: 591 tests and subtests pass, and each of the 531 that fail does so only because the engine binary is absent (a script checked each failing test's output for that);test/docs, withoutTestEngineLinksResolve, which needs the engine binary: 7 of 7 pass.go vetoverinternal/renderandtest/docs, and the sign-off gate over all thirteen commits. No type, marker or generator input changed, so there was nothing to regenerate.Positionsagain and 18 new, match it, and still match the engine change's pushed head, crewlet/crewlet@b5dd4bb53;(*FleetStore).eachUnderand a second read ingetLatestmoved no row the table held before these two commits (Positions,Floor,MaintenanceandAgreeObjectBackendstill matched), and the new rows for those two functions catch them;settled's timeout doubled, a read back added insettledBucket.Update, a second decide insettle,Decide's fallback doubled) moved none of the other 194 rows, and each of the four rows for those functions catches its own.At a7d9b93, after commits 14 and 15:
go test -race -count=1overinternal/renderandtest/docs: 599 tests and subtests pass, and each of the 556 that fail does so only because the engine binary is absent (a script checked each failing test's output for that),TestMirroredEngineDeclarationsAreUnchanged,TestCredentialPositionsAreTheEnginesMarks, the contract tests andTestEngineLinksResolveamong them. Among the passes:TestEngineConfigCannotSetAKeyAFieldRenders,TestCompanyGolden,TestCompanyLeavesItsInputAlone,TestBootstrapGolden,TestAMirroredDeclarationThatMovedFails,TestTheDocsStateTheBases,TestEveryFindingHasABasisandTestStreamFloorFiguresAreTheEngines, and the docs gate'sTestRelativeLinksResolveandTestEveryPageIsInTheIndex.go vetoverinternal/render,test/crdandtest/docs, and over theapimodule, which compiles the edited envtest cases without running them;make verify-generated, which leaves the tree unchanged; and the sign-off gate over all fifteen commits.armInboxRelay, 195 of the 198 rows match,armInboxRelayamong them. The other three are under "Expected red" below.At 4df7827, after commits 16 and 17:
go test -race -count=1overinternal/render, the whole package: 592 tests and subtests pass, and the 555 that fail do so only because the engine binary is absent,TestMirroredEngineDeclarationsAreUnchangedamong them. Each of the 531 failing leaf tests says so in its own output (a script read each), and nothing fails for any other reason.TestAMirroredDeclarationThatMovedFailspasses, and passed again on review.internal/renderon review;go vetoverinternal/render; and the sign-off gate over all seventeen commits. No type, marker, generator input or page changed, so there was nothing to regenerate.LaunchandLaunchExecutoramong them. The other three are under "Expected red" below.Launch's new digest holds from crewlet/crewlet@521bd300f to that head, andLaunchExecutor's from crewlet/crewlet@86267fe2c. At crewlet/crewlet@a2634a307, where both rows were read before, the same check gives back their old digests.At 2b1e20e, after commits 18 to 20 (the engine change's pushed head, crewlet/crewlet@2887e830c, read from git objects copied into a temporary directory and deleted after):
make manifests generate api-docs chart, whose output is committed;go build ./...andgo vetin both modules, which compilestest/crdandinternal/controller; and gofmt.ValidateCustomResourceDefinition: no errors. It also measured the new rule's cost: 17 per name, which is 8 704 on a setup step'senvitems and 544 on a Seat'sspec.sandbox.env. The setup block's other rules stay at 6 582 608 of 10 000 000.go test -race -count=1 ./internal/render/, at commit 20's first version: 78 top-level tests pass, and each of the 40 that fail does so only because the engine binary is absent.TestAMirroredDeclarationThatMovedFailspasses. The three rows these commits record match the engine at that head:(*launcher).Launch,ReadBoxURLsandBoxURLs.All.ReservedSandboxEnvnames exactly the five variables. The engine'svalidateReservedSandboxEnvrefuses them inproviders.sandbox.setup[i].env,roles[i].sandbox.envandroles[i].sandbox.setup[j].env, the paths thetest/crdcases expect. ADR-0063 and the heading the credentials guide links exist.Launchstill assembles the run's environment, the coding run and the sandbox's credentials before it resolves the repositories and asks the coordinator to launch.go vetoverinternal/engine/enginetestandinternal/render.TestAMirroredDeclarationThatMovedFailsandgo test ./internal/engine/enginetest/did not run again: their run waited more than 30 minutes behind other jobs on the shared runner and was cancelled. The fix changes no digest, decision text or behaviour.At 9341b49, after commits 21 to 24 (the engine change's pushed head, crewlet/crewlet@ab6d97e67, read from source extracted into a temporary directory and deleted after):
LaunchExecutoramong them, with the new digest commit 22 records. The other seven are under "Expected red" below. Between the previous read (crewlet/crewlet@4ee3f06a5) and that head,LaunchExecutormoved only with crewlet/crewlet@0645a8eb5.make manifests generate chart api-docs, whose output is committed; gofmt overapi,internalandtest;go build ./...;go vetoverinternal/controller,internal/engineandinternal/render, and in theapimodule.go test -count=1 ./internal/render/, at commit 22: 78 top-level tests pass, and the 40 that fail do so because the engine binary is absent (21 checked directly; the other 19 are parents of engine-contract subtests).go test -race -count=1over the controller's pause unit tests at 9341b49: the newTestAStoppedSeatSaysItsRunsWritesAreRefused(a stop, and a pause only),TestAnOwnedPauseFollowsTheRecordNotTheRoster,TestAPauseThatCannotBeMadeIsAskedForAgainSoonandTestASeatTheEngineDoesNotRunHasNoPausepass.config_pause.goput back to its old wording, the new test fails on both the condition's message and the asked-for state.config_pause.goor the engine client'sseats.go.What did not run here, left to CI:
test/crdand the controller's envtest suite,TestReservedRunVariablesValidationandTestReservedRunVariablesFollowTheEngineamong them; the engine-backed tests (the contract tests,TestMirroredEngineDeclarationsAreUnchanged, the engine client andsecretsync);test/chartandtest/e2e;make lintand helm-lint; andmake checkas a whole.Expected red until the engine change lands
CI runs Crewlet's
mainimage, which does not have the engine change yet. So the engine-backed jobs fail on this branch until feat(code): host the company's git repositories inside the engine crewlet#298 merges and the image rebuilds; re-run CI then.Against the engine change's pushed head (crewlet/crewlet@ab6d97e67),
TestMirroredEngineDeclarationsAreUnchangedfails on seven rows. Three,(*Engine).join,startStateLogandbootSecrets, this repository'smainalready records at fix(engine): read secrets and build the company after a node recovers crewlet#311, which the engine change has not taken yet:bootSecretsis not on its branch at all, and it leavesjoinandstartStateLogas Crewlet'smainhad them, so they match once it takesmain. The other four moved with later changes on that branch, and the commits that follow them here read them again: the estate service'sServe(the git write bound and the git lanes),(*stateLog).start, and the embedded broker'sconnectandawaitLeafReady(the in-process clients and the fleet's keys). Like the rest of the table, every row is read again against the engine change's final head before this leaves draft.That head also refuses the backends this branch renders for the engine's own tracker and knowledge base. The branch is based on this repository's
main, which follows Crewlet'smainin calling them Worklet and Doclet, renderedworkletanddoclet(crewlet/crewlet@4cbb2ad59). The engine change has not taken that rename yet and still calls bothnative, so it refuses thenimbusandorg-chartgoldens, and every Company left on the default tracker, as naming a backend it does not know. Until the engine change takes Crewlet'smain, which it does once, at its end, the engine-backed suites cannot pass against either engine on its own: Crewlet'smainhas no code host, and the engine change lacks the rename. Commit 14 already follows the merged engine: the horizon leaves the tracker's block, which Crewlet'smaincallstracker.workletand the engine change calledtracker.native, for the top-levelinbox.This repository's
mainhas moved on. The branch is based onmainat 7764ebe.mainhas since taken feat(api): turn on the company's dashboard assistant #31 (the dashboard assistant), feat(api): rename the LLMProvider modes Text and Agent to Host and Box #35 (the LLMProvider modes renamedHostandBoxafter feat(toolloop): serve a phase's tools to its CLI as one MCP server crewlet#371), feat(controller): run box-mode seats in seat pools through the bridge #36 (box-mode seats in seat pools, through the bridge), fix(controller): keep every value an engine pod may run or start on #38 (the revision each node runs), feat(api): let a FullAccess seat opt in to GitHub workflow writes #39 (aFullAccessseat's opt-in to GitHub workflow writes, after feat(engine): agents push and open pull requests as their own GitHub App crewlet#395), feat(api): run every agent seat's executor as its coding CLI's own run #40 (every agent seat's executor run as its coding CLI's own run, after feat(engine): run every seat's executor as its coding CLI's own run crewlet#388), test(render): record carryOnLost as read where a run's ask is on its row #41 (carryOnLostread again where a run's ask is on its row), feat(api): sign a model provider's CLI in with its env under auth mode Env #42 (a model provider's CLI signed in with its env under auth modeEnv), feat(secretsync): a sync is one activation, and a failed one activates next time #49 (a secret sync as one activation), test(engine): wait for the /fleet record of an apply as for /health #50 (the engine harness waiting for an apply's/fleetrecord as for/health), feat(seat): per-agent integration access #51 (every agent's integration access as part of the Seat) and docs: name the assistant Ask Crewlet #53 (the dashboard assistant named Ask Crewlet). The branch conflicts withmain(now 3a767f7) in 35 files. 24 are the ones it conflicted in before feat(seat): per-agent integration access #51: the EngineClass and Company types with their CRDs, chart and API reference, and the Seat's CRD and its chart copy; the boot windows and their tests; the engine harness; the render'sorg.go, its company test table andgithub-enterprisegolden input, the mirror table and the stream floor test; thetest/crdcases, the Company's among them since commit 14; and the runtime, troubleshooting and company-document pages. Commits 18 to 20 add three: the Seat's types, the credentials guide and the render'scodework_test.go. feat(seat): per-agent integration access #51 adds eight: the sample Seat, the installing guide, the render'scompany.go,diff.go,document.goandenums.goand itssecret-positionsgolden input, andtest/crd's org-chart cases. Six things follow from that:mainruns state logs this branch does not count: the Dashboard Assistant's, and since feat(seat): per-agent integration access #51 the chat log (crewlet#142). This repository'smainnow counts them in a data pod's boot (1866 s, or 2006 s with S3) and in the volume floors (14Gi with NATS, 11Gi with S3). The engine change adds the code log. Once it takes Crewlet'smain, the windows, the volume floors and the harness are counted again for every log that engine runs.Agent.mainrecords at Crewlet commits the engine change has not taken yet, from feat(toolloop): serve a phase's tools to its CLI as one MCP server crewlet#371 on, join the expected red above until it takes them.spec.integrations.github.permissions), and feat(api): let a FullAccess seat opt in to GitHub workflow writes #39'sworkflowsis now one of those permissions. Commit 1 moves the same tier tospec.code.access, after the engine change'srole.code.access. So the merge follows whatever the engine change has, once it takes Crewlet'smain: a seat's code access beside its GitHub permissions, or one of them alone. Its rules,test/crdcases, goldens and guides follow, with no alias for either shape.maina setup step'senvis stillSecretEnvVar. The merge keepsSandboxEnvVarfor it, and the reserved-name rule on it and onSeatEnvVar.mainno longer has the run_sandbox launch that commit 16 reads again: feat(engine): run every seat's executor as its coding CLI's own run crewlet#388 runs every executor as its coding CLI's own run, and feat(api): run every agent seat's executor as its coding CLI's own run #40 here follows it. Somain's table has no(*launcher).Launchrow, and readsLaunchExecutorfor decisions of its own: the member a launch picks, the cell it places the run in, and the bridge address that cell's box can dial. The engine change has not taken feat(engine): run every seat's executor as its coding CLI's own run crewlet#388 yet. Once it has, the merge here readsLaunchExecutoragain at that head for both feat(api): run every agent seat's executor as its coding CLI's own run #40's decisions and commit 17's, and keeps commit 16'sLaunchrow only if that engine still has the launch.The branch takes
mainagain, and is read again against the engine change's final head, before it leaves draft.Checklist
make checkpasses. Not yet: its unit and static parts ran here (above). The envtest, engine and chart suites and lint run in CI, which needs feat(code): host the company's git repositories inside the engine crewlet#298 merged and themainimage rebuilt first.the API server enforces it (a CEL rule refuses what it must refuse). Every new rule has a
test/crdcase, which runs in CI.make manifests generate chart api-docsdocs/) for any change to the API, behaviour, thedeployment or a make target
value, and no value reaches a pod template, a status, an event or a log. The engine change adds and removes no credential position.
newest release and pinned exactly, or carries the reason it is held
back at the pin (none was)
git commit -s; CI'ssign-offjob failswithout it
type(scope): summaryit is what appears in the generated release notes
🤖 Generated with Claude Code