Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cmd/volume-agent/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
//
// The data path is QEMU's. This process prepares each volume's local qcow2 chain, hands the
// paths to whoever launches the VM, and speaks QMP to the QEMU that ends up there — it does
// not start one; internal/qcow carries the two-path contract. So: claim the data directory,
// not start one; qcow carries the two-path contract. So: claim the data directory,
// read the key, register, heartbeat, hold a lease, learn which volumes to serve, prepare a
// chain for each, and report what it saw.
package main
Expand All @@ -32,10 +32,10 @@ import (
"github.com/spin-stack/storage/internal/descriptor"
"github.com/spin-stack/storage/internal/obs"
"github.com/spin-stack/storage/internal/publisher"
"github.com/spin-stack/storage/internal/qcow"
"github.com/spin-stack/storage/internal/recovery"
"github.com/spin-stack/storage/internal/simio/real"
"github.com/spin-stack/storage/internal/storecfg"
"github.com/spin-stack/storage/qcow"
)

// version is the build identity the Agent reports. Overridden at link time with
Expand Down
28 changes: 14 additions & 14 deletions hack/comment-rot-allow.txt
Original file line number Diff line number Diff line change
Expand Up @@ -59,19 +59,19 @@ internal/agent/loop.go:Loop.giveUpWhatIsNoLongerOurs:lease-valid # "not 'the lea
internal/metadata/metadata.go:Store.Now:lease-valid # about the Control Plane granting an epoch "while the old writer's monotonic lease is still valid" — mutual exclusion, the one part of fencing V1 keeps

# --- naming a withdrawn mechanism to say it is gone, and two forward references --------
internal/qcow/chain.go:overlay:compaction # "§19 already owns flattening, as compaction, outside the recovery path" — §21's chain compaction, which internal/qcow/compact.go performs. The comment is why a recovery does NOT flatten.
internal/qcow/compact.go:MaxLayers:compaction # the ceiling's reason: §19's compaction is what keeps a chain an order of magnitude below it, and it runs — DefaultCompaction is what a binary uses
internal/qcow/manager.go:DefaultRotateAtBytes:remote # "a slow remote one at F = 200 ms" is a point on the backend curve the threshold is derived over, not the withdrawn remote WAL
qcow/chain.go:overlay:compaction # "§19 already owns flattening, as compaction, outside the recovery path" — §21's chain compaction, which qcow/compact.go performs. The comment is why a recovery does NOT flatten.
qcow/compact.go:MaxLayers:compaction # the ceiling's reason: §19's compaction is what keeps a chain an order of magnitude below it, and it runs — DefaultCompaction is what a binary uses
qcow/manager.go:DefaultRotateAtBytes:remote # "a slow remote one at F = 200 ms" is a point on the backend curve the threshold is derived over, not the withdrawn remote WAL
internal/controlplane/clone.go:MaxChainDepth:compaction # the ceiling's cost is per commit of the ancestor, and §21's compaction is what bounds an ancestor's growth — the same subject, on the Control Plane's side.
internal/controlplane/clone.go:Clone:compaction # two constraints on §19's compaction: what it must leave true about a clone's depth, and that a clone mints a fresh DEK or breaks crypto-shred.
internal/qcow/compact.go:CompactionPolicy.AtLayers:compaction # §21's chain compaction, which this file performs. The withdrawn "compaction" the checker knows is v5's objectization of WAL extents, a different subject that shares a word.
internal/qcow/compact.go:CompactionPolicy.Validate:compaction # same subject: the thresholds §21 says must be measured rather than chosen.
internal/qcow/compact.go:Manager.compact:gc # same subject: what a collapse deliberately does not delete, and whose job that is instead.
internal/qcow/compact.go:Manager.recordCollapsed:compaction # same subject: why a collapse leaves the RPO anchor where it is — it publishes nothing the guest wrote.
internal/qcow/compact.go:compaction:compaction # same subject: the package comment naming what this file is for.
internal/qcow/chain.go:Paths.Rename:compaction # same subject: the one caller of Rename is the flattened root a collapse builds under a temporary name.
internal/qcow/chain.go:checkNotStale:compaction # same subject: a HEAD this host's own collapse published is not a chain another host moved past.
internal/qcow/sweep.go:sweep:compaction # same subject: the half-written image a killed convert leaves, which is the one non-layer file the sweep takes.
internal/qcow/manager.go:Config.Compaction:compaction # same subject: the policy an operator sets, zero being "collapse nothing until somebody has measured".
internal/qcow/manager.go:Manager.reconcile:compaction # same subject: where a collapse runs in the cycle, and why it runs for an unattached volume too.
internal/qcow/sweep.go:LiveImage:compaction # §21's collapse, which is live: the sentence is about the `.compacting` file a killed `qemu-img convert` leaves, and why the sweep does take that one
qcow/compact.go:CompactionPolicy.AtLayers:compaction # §21's chain compaction, which this file performs. The withdrawn "compaction" the checker knows is v5's objectization of WAL extents, a different subject that shares a word.
qcow/compact.go:CompactionPolicy.Validate:compaction # same subject: the thresholds §21 says must be measured rather than chosen.
qcow/compact.go:Manager.compact:gc # same subject: what a collapse deliberately does not delete, and whose job that is instead.
qcow/compact.go:Manager.recordCollapsed:compaction # same subject: why a collapse leaves the RPO anchor where it is — it publishes nothing the guest wrote.
qcow/compact.go:compaction:compaction # same subject: the package comment naming what this file is for.
qcow/chain.go:Paths.Rename:compaction # same subject: the one caller of Rename is the flattened root a collapse builds under a temporary name.
qcow/chain.go:checkNotStale:compaction # same subject: a HEAD this host's own collapse published is not a chain another host moved past.
qcow/sweep.go:sweep:compaction # same subject: the half-written image a killed convert leaves, which is the one non-layer file the sweep takes.
qcow/manager.go:Config.Compaction:compaction # same subject: the policy an operator sets, zero being "collapse nothing until somebody has measured".
qcow/manager.go:Manager.reconcile:compaction # same subject: where a collapse runs in the cycle, and why it runs for an unattached volume too.
qcow/sweep.go:LiveImage:compaction # §21's collapse, which is live: the sentence is about the `.compacting` file a killed `qemu-img convert` leaves, and why the sweep does take that one
2 changes: 1 addition & 1 deletion hack/comment-rot.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Which production comments still describe a mechanism this tree withdrew — computed,
# not remembered.
#
# A finding is a (symbol, term) pair — internal/qcow/chain.go:Open:checkpoint. Rejected
# A finding is a (symbol, term) pair — qcow/chain.go:Open:checkpoint. Rejected
# keys: file:line (moves on any edit above it), the whole file (collapses a file's eight
# mentions into one decision), the line's text hashed (a reword fails the gate with a
# message about a hash).
Expand Down
4 changes: 2 additions & 2 deletions hack/deadcode-allow.txt
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ internal/lifecycle.RefusalNames # compared with volumes.refusal's CHECK list
# --- the observability test provider ----------------------------------------------------
# obs.NewTestProvider is why every metric in this repository is proven by reading the
# collected value rather than by checking that a recorder was called. It cannot live in a
# _test.go file: internal/agent, internal/qcow and internal/obs's own tests all import it.
# _test.go file: internal/agent, qcow and internal/obs's own tests all import it.
internal/obs.NewTestProvider # the in-memory meter provider tests collect from; production wires the OTLP one
internal/obs.Provider.CollectedMetrics # reading it is how a test asserts on what left the process
internal/obs.Provider.GaugeValues # same, for gauges
Expand Down Expand Up @@ -82,4 +82,4 @@ package internal/simio/network # harness-only: the DST partition scenario drive
# it agreeing with the scenario and with nothing else, which is the shape a checker cannot
# catch — so the production function is exported and a binary reaches it through
# Manager.Apply.
internal/qcow.Sweep # driven by internal/dst; production reaches the same code through Manager.Apply
qcow.Sweep # driven by internal/dst; production reaches the same code through Manager.Apply
1 change: 0 additions & 1 deletion hack/refs-allow.txt
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ hack/analyzers/simulable/simulable.go internal/testinfradriver # deliberately h
docs/plan/DECISIONS/ADR-0026-v1-accepts-session-rpo.md internal/wal # an ADR records the decision as it was taken; the package it names is what the decision was about

# --- other repositories -----------------------------------------------------------------
internal/qcow/chain.go cmd/runner # spin's runner, which launches the VMs (ADR-0021). Not ours to resolve.
docs/plan/DECISIONS/ADR-0021-integration-with-spin.md cmd/cli # spin's operator CLI, the reason storage ships no volctl

# --- a regex that reads as a path --------------------------------------------------------
Expand Down
4 changes: 2 additions & 2 deletions integration/e2e/compaction_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ import (
storagev1 "github.com/spin-stack/storage/api/gen/spin/storage/v1"
"github.com/spin-stack/storage/internal/commit"
"github.com/spin-stack/storage/internal/ids"
"github.com/spin-stack/storage/internal/qcow"
"github.com/spin-stack/storage/internal/simio/real"
"github.com/spin-stack/storage/internal/simio/sim"
"github.com/spin-stack/storage/internal/testinfra"
"github.com/spin-stack/storage/qcow"
)

// The volume this lane collapses. Small, because every layer of it is written in full.
Expand Down Expand Up @@ -104,7 +104,7 @@ func TestACompactedRootReadsBackAsTheChainItReplaced(t *testing.T) {
Disk: sim.NewDisk(),
// No VM: the QMP socket does not exist, so the volume is unattached and the
// collapse can finish in one cycle. A rebase under a running guest is what v6 §5
// forbids, and internal/qcow's own tests hold that end.
// forbids, and qcow's own tests hold that end.
Runner: real.NewRunner(), Paths: files, Dialer: real.NewUnixDialer(),
Recovery: noHistory{}, Publisher: pub,
})
Expand Down
2 changes: 1 addition & 1 deletion integration/e2e/datadir_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import (
//
// The lock has moved twice and this test is why the moves were safe: into `main` when the
// volume manager that used to take it was withdrawn, and back into the volume manager
// (internal/qcow) when Stage 1 gave it a directory layout to own. What it asserts is
// (qcow) when Stage 1 gave it a directory layout to own. What it asserts is
// about the *process*, not about which type inside it holds the descriptor — which is
// exactly the property that let the second move happen without a second argument.
//
Expand Down
2 changes: 1 addition & 1 deletion internal/agent/agent.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
// device, and the set of volumes this host is serving. cmd/volume-agent is the only
// place the real implementations are constructed.
//
// The volume manager behind the VolumeSource is internal/qcow: QEMU owns the local
// The volume manager behind the VolumeSource is qcow: QEMU owns the local
// copy-on-write format, and what is left in this package is the half that talks to the
// Control Plane. VolumeReconciler is the seam, qcow.Manager plugs into it, and VolumeSet
// is what the loop's own tests drive.
Expand Down
2 changes: 1 addition & 1 deletion internal/dst/checkers_reconcile.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ func reconcileCheckers() []Checker {
// A publish naming a layer no seal event named is not this checker's subject and is
// skipped: `two-hosts-cannot-both-publish` publishes layers that were never rotated into
// being, and publishing a layer that is *not* sealed — the tip, with a guest still
// writing into it — is asserted through the Manager in internal/qcow's reconcile
// writing into it — is asserted through the Manager in qcow's reconcile
// adversary lane, where the publisher can see which file it was handed.
type SealOrderChecker struct {
sealed []string
Expand Down
2 changes: 1 addition & 1 deletion internal/dst/planted_bug_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ func TestSingleWriterCheckerCatchesAdvisoryPreconditions(t *testing.T) {
//
// Each was written, run, and watched go red before the line that closes it existed. The
// second is the sharp one and it is still checkable in one edit: delete the
// `keep[LayerIDOfImage(open.Path)]` line in internal/qcow/sweep.go and
// `keep[LayerIDOfImage(open.Path)]` line in qcow/sweep.go and
// `sweepScenario(true)` removes the layer a guest is writing into, which is how that line
// came to be there.
//
Expand Down
6 changes: 3 additions & 3 deletions internal/dst/scenarios_reconcile.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ import (
"github.com/spin-stack/storage/internal/commit"
"github.com/spin-stack/storage/internal/crypto"
"github.com/spin-stack/storage/internal/ids"
"github.com/spin-stack/storage/internal/qcow"
"github.com/spin-stack/storage/internal/simio/objectstore"
"github.com/spin-stack/storage/qcow"
)

// reconcileScenarios is the half of the reconciler a simulation can reach.
Expand All @@ -19,7 +19,7 @@ import (
// chain, ask a live QEMU which file it has open, snapshot a new overlay over the tip —
// is another process's work, and a fake for it here would be a second implementation of
// qemu-img: a model of a program, agreeing with the scenario and with nothing else. That
// half is tested against the real binary, in internal/qcow's adversary lane
// half is tested against the real binary, in qcow's adversary lane
// (adversary_reconcile, adversary_crash-points, adversary_qcow-restart) and end to end by
// `task demo:stage2`.
//
Expand Down Expand Up @@ -61,7 +61,7 @@ const reconcileVirtualSize = 1 << 30
// coming up and the power failure is acknowledged and not persisted, which is what a
// device with a volatile write cache and no flush does; the power failure then takes them
// all. Deliberately not from the very first write: a state.json that was never durable at
// all comes back zero-length, and ReadState refuses that outright (internal/qcow's state
// all comes back zero-length, and ReadState refuses that outright (qcow's state
// table covers it). What this reaches is the dangerous shape instead — a record that is
// framed, digest-sound, believable, and one power failure out of date.
func reconcileScenario(volatileCache bool) Scenario {
Expand Down
4 changes: 2 additions & 2 deletions internal/dst/scenarios_sweep.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"fmt"

"github.com/spin-stack/storage/internal/ids"
"github.com/spin-stack/storage/internal/qcow"
"github.com/spin-stack/storage/qcow"
)

// sweepScenarios drives the one rule in this system that deletes files.
Expand All @@ -14,7 +14,7 @@ import (
// records and the pointers, and both arrive through I/O this package can break. What it
// cannot reach is the live QEMU: `Manager.Apply` asks a running guest which image it has
// open and refuses a volume it cannot account for, and a fake for that here would be a
// model of qemu-img agreeing with the scenario. That half is internal/qcow's.
// model of qemu-img agreeing with the scenario. That half is qcow's.
//
// The fault is a disk that acknowledges an fsync it does not honour. It is aimed here
// rather than anywhere else because the sweep's keep-set comes from the records, so a
Expand Down
2 changes: 1 addition & 1 deletion internal/lifecycle/lifecycle.go
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,7 @@ func (r CordonReason) OverwritableNames() []string { return names(cordonOverwrit
// `err.Error()`, which embeds a volume id and a path, so no two rows compare equal, the
// -fleet-status column becomes a vocabulary nobody controls, and the first alert written on
// it matches a substring. Every value here is a decision at a named line in internal/agent
// or internal/qcow, so extending the vocabulary is the same commit. The sentence an operator
// or qcow, so extending the vocabulary is the same commit. The sentence an operator
// needs — which layer, which key — rides alongside as free text nothing branches on
// (metadata.VolumeProgress.RefusalDetail).
//
Expand Down
2 changes: 1 addition & 1 deletion internal/publisher/metrics_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ func TestPublishRecordsWhatLeftTheHost(t *testing.T) {
if sent[labels] != stored.Size {
t.Errorf("layer_upload_bytes_total = %d, want %d", sent[labels], stored.Size)
}
// The plaintext is the number this must *not* be: it is what internal/qcow measures
// The plaintext is the number this must *not* be: it is what qcow measures
// locally, and reporting it here would understate the bucket by a tag per frame.
if stored.Size == int64(len(w.plain)) {
t.Fatal("the sealed object is the same length as the plaintext; this test proves nothing")
Expand Down
6 changes: 3 additions & 3 deletions internal/publisher/publisher.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Package publisher turns a sealed layer on this host's disk into a published commit.
//
// internal/qcow owns a volume's local chain and decides *when* a layer is ready; this is
// qcow owns a volume's local chain and decides *when* a layer is ready; this is
// *what* happens then — fetch the volume's key, read the file, and run v6 §9's publish
// protocol. It needs a Control Plane and an object store; the chain needs neither.
package publisher
Expand All @@ -16,9 +16,9 @@ import (
"github.com/spin-stack/storage/internal/commit"
"github.com/spin-stack/storage/internal/crypto"
"github.com/spin-stack/storage/internal/obs"
"github.com/spin-stack/storage/internal/qcow"
"github.com/spin-stack/storage/internal/simio/clock"
"github.com/spin-stack/storage/internal/simio/objectstore"
"github.com/spin-stack/storage/qcow"
)

// Keys hands over a volume's wrapped key material. It is agent.Loop in production: the
Expand Down Expand Up @@ -81,7 +81,7 @@ func (p *Publisher) Publish(ctx context.Context, l qcow.SealedLayer) error {
return err
}
// The sealed length, which is what left this host and what the bucket is charged
// for; the qcow2's own length is the local number and internal/qcow records it.
// for; the qcow2's own length is the local number and qcow records it.
volume := obs.String("volume", l.VolumeID)
p.rec.Observe(ctx, "layer_size_bytes", float64(m.Layer.SizeBytes), volume)
p.rec.Count(ctx, "layer_upload_bytes_total", m.Layer.SizeBytes, volume)
Expand Down
2 changes: 1 addition & 1 deletion internal/publisher/publisher_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ import (
"github.com/spin-stack/storage/internal/crypto"
"github.com/spin-stack/storage/internal/ids"
"github.com/spin-stack/storage/internal/publisher"
"github.com/spin-stack/storage/internal/qcow"
"github.com/spin-stack/storage/internal/simio/sim"
"github.com/spin-stack/storage/qcow"
)

// fakeKeys is the Control Plane's half: it hands over a volume's wrapped DEK, or refuses
Expand Down
2 changes: 1 addition & 1 deletion internal/qmp/qmp.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
//
// # Why there is a client here at all
//
// The Agent does not run QEMU (see internal/qcow): it prepares a volume's chain and has no
// The Agent does not run QEMU (see qcow): it prepares a volume's chain and has no
// other way to know whether anything is using it. QMP is the only channel that answers, and
// v6 §7 makes it the mandatory one — flush, snapshot, switch to the new tip, and confirm QEMU
// is using it.
Expand Down
2 changes: 1 addition & 1 deletion internal/recovery/adversary_blank-disk_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ import (
"sync"
"testing"

"github.com/spin-stack/storage/internal/qcow"
"github.com/spin-stack/storage/internal/recovery"
realio "github.com/spin-stack/storage/internal/simio/real"
"github.com/spin-stack/storage/qcow"
)

// TestAdversaryAHeadThatNamesNoCommitCrashesTheRestore.
Expand Down
2 changes: 1 addition & 1 deletion internal/recovery/adversary_corrupt-layer_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ import (
"testing"

"github.com/spin-stack/storage/internal/commit"
"github.com/spin-stack/storage/internal/qcow"
"github.com/spin-stack/storage/internal/recovery"
"github.com/spin-stack/storage/qcow"
)

// rottenRunner is a qemu-img that opens the file before it answers about it.
Expand Down
2 changes: 1 addition & 1 deletion internal/recovery/adversary_formats_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ import (

"github.com/spin-stack/storage/internal/commit"
"github.com/spin-stack/storage/internal/framed"
"github.com/spin-stack/storage/internal/qcow"
"github.com/spin-stack/storage/internal/recovery"
"github.com/spin-stack/storage/internal/simio/objectstore"
"github.com/spin-stack/storage/qcow"
)

// putFramed writes a hand-built structural object the way a writer of this format would:
Expand Down
2 changes: 1 addition & 1 deletion internal/recovery/adversary_rebuild_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"testing"

"github.com/spin-stack/storage/internal/ids"
"github.com/spin-stack/storage/internal/qcow"
"github.com/spin-stack/storage/qcow"
)

// TestAdversaryALayerIdThisHostHoldsForAnotherVolumeIsNotStoodInForThisOne.
Expand Down
Loading
Loading