Skip to content

sam-one - #360

Merged
aojea merged 10 commits into
google:mainfrom
aojea:samone
Sep 6, 2026
Merged

sam-one#360
aojea merged 10 commits into
google:mainfrom
aojea:samone

Conversation

@aojea

@aojea aojea commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

is the all-in-one SAM distribution: control plane, libp2p router
and storage in a single binary serving a single public port.

aojea added 10 commits September 4, 2026 16:09
Init bootstraps the signing keyring, discovers OIDC providers and starts
the key-rotation loop; RegisterRoutes registers every handler on a caller
owned mux. Start composes them, so cmd/sam-control-plane is unchanged.
A single-binary distribution embedding the control plane calls
Init + RegisterRoutes and never binds the control plane's own listener.
Pinned by TestInitRegisterRoutesEmbedded, including empty-issuer boot and
Close without Start.
Options.HTTPFallbackHandler, when set, serves ordinary HTTP requests
arriving on the router's /ws listen addrs so one public TCP port carries
both libp2p and REST traffic. The transport list is assembled explicitly
in that mode because combining DefaultTransports with an extra
Transport(websocket.New, ...) registers the WS transport twice and fails
host construction.
TestSinglePortWebSocketAndHTTP pins the whole topology: an OIDC-less
auto-approve control plane, a router self-enrolled over loopback with a
store-seeded bootstrap token, and control-plane routes plus a libp2p dial
both served through the router-owned port.
internal/standalone wires the shared SQL store, the control plane on a
loopback-only listener, and the router owning the public port: WebSocket
upgrades become libp2p connections while other HTTP requests fall through
to the control-plane mux. The embedded router keeps its stock enrollment,
lease and keys-sync flows against loopback, authenticated by a per-boot
single-use bootstrap token, so no component grows in-process shortcuts.

First boot provisions everything a zero-config mesh needs: signing
keyring, persisted admin and join tokens (0600 files in --data-dir), and
a mesh policy seed, either from --policy-file (protojson, first boot
only, DB authoritative afterwards) or an explicitly-logged open
development default. --external-url derives the advertised ws/wss
multiaddr for platforms that terminate TLS at the edge (Cloud Run).

cmd/sam-one is the cobra entrypoint with SAM_TOKEN / SAM_ADMIN_TOKEN /
SAM_EXTERNAL_URL env fallbacks and a quickstart banner.

TestStandaloneNodeJoin pins the CUJ: first boot, then a stock sam-node
enrolls with the generated join token and connects to the embedded
router over WebSocket through the single public port, in under a second.
Assets move to internal/console/public so go:embed works in-package; the
console serves them from the embedded FS by default while --static-dir
keeps overriding from disk for the ui-dev live-edit workflow. sam-one
mounts the console at /console on the shared single-port mux, constructed
after the listener is live so its /info discovery succeeds over loopback.
token create/list and admin ban are thin clients of the running server's
/admin endpoints authenticated with the admin token (flag, SAM_ADMIN_TOKEN,
or the token persisted in --data-dir), so a second process never opens the
SQLite file. /admin/bootstrap-tokens gains an admin-gated GET returning
the stored records to back token list.
Dockerfile.sam-one follows the distroless nonroot + /data-seed pattern and
pins a stable container port via CMD; goreleaser gains the sam-one build.
--bind-address (host) and --port are now independent flags with the port
defaulting to 0: a free port is picked and published in the startup banner
like the generated tokens, instead of baking any platform's port contract
into the code.
tests/e2e/standalone.bats boots the real binary on a random port, parses
the banner, joins two real nodes, and pushes a request from node B through
its egress proxy across the router to a smoke HTTP service registered on
node A - pinning the single-port dataplane end to end, plus the token
create/list CLI against the live server.
Behind a TLS-terminating proxy or NAT (verified on Cloud Run) every peer
and every fallback HTTP request shares a handful of source IPs, and three
stacked per-IP defaults in libp2p's resource manager each take the whole
single-port listener down under normal reconnect churn: the inbound
connection cap (8), the connection rate limit (0.2/s, burst 16 - the
denials surface as 'rate limit exceeded' and self-heal as the bucket
refills, which on Cloud Run reads as minutes of 429 'no available
instance'), and the small transient scope that zombie conns from killed
peers pool in.

Options.ConnsPerSourceIP (sam-router: --conns-per-source-ip; 0 keeps the
libp2p defaults) scales all three together: the per-subnet conn caps, the
per-subnet rate limiter (same shape as upstream's, loopback stays exempt)
and the system/transient scope floors. sam-one pins it to the connection
manager's high watermark so the global cap is what binds.

Root-caused with tcpdump (server SYN-ACKs, reads the request, then RSTs
while the app and kernel counters look healthy) plus rcmgr debug logs;
reproduced and regression-tested by churning relay-only nodes against a
containerized sam-one: 132/160 failed probes before, 0/160 after.
Joining a mesh now always requires an explicit control plane (or the one
already stored from a previous enrollment): the interactive public-mesh
chooser is gone, 'sam-node join' without a URL errors out, 'run --join'
demands --control-plane, and the mobile FFI refuses to start unenrolled
without a configured ControlPlaneURL instead of pointing itself at the
testnet. The testnets keep running; nodes just never end up on them by
default.
Verified end to end: single-port wss enrollment, a relayed service call
between two NAT-hidden nodes through the Cloud Run router, and the admin
CLI against the public URL. Documents the load-bearing flags (singleton,
no CPU throttling, request timeout), the reserved /healthz path, ephemeral
state, and rollout overlap behavior.
Operator knobs forward to the components under --control-plane-* and
--router-* prefixes (lease/rotation/grace/biscuit-ttl, manual enrollment,
sync intervals, watermarks, per-source-IP budget, DHT record lifetimes,
loopback advertisement). Structural wiring (loopback CP listener, shared
store, derived ws listener) deliberately stays non-configurable; zero
values keep each component's defaults.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces sam-one, an all-in-one standalone server distribution that combines the control plane, libp2p router, and storage into a single binary serving a single public port. It also embeds the web console assets directly into the binary and updates sam-node to require an explicit control plane URL when joining a mesh. The review feedback recommends implementing a deferred cleanup block in standalone.Start to prevent resource leaks on initialization failures, and exposing the ConnsPerSourceIP setting as a configurable option and CLI flag in sam-one to allow customization of connection limits behind proxies.

Comment on lines +172 to +176
func (o *Options) Validate() error {
if _, err := wsListenMultiaddr(o.BindAddress); err != nil {
return fmt.Errorf("invalid bind address %q: %w", o.BindAddress, err)
}
if o.ExternalURL != "" {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If Start fails after initializing the database store or the control plane, these resources (including background goroutines, HTTP servers, and database connections) are leaked if standalone is used as an embedded library. Adding a deferred cleanup block that calls s.Close() on failure ensures all partially allocated resources are cleanly released.

Suggested change
func (o *Options) Validate() error {
if _, err := wsListenMultiaddr(o.BindAddress); err != nil {
return fmt.Errorf("invalid bind address %q: %w", o.BindAddress, err)
}
if o.ExternalURL != "" {
store, err := storage.NewSQLStore(s.opts.DBDriver, s.opts.DBDSN)
if err != nil {
return fmt.Errorf("failed to open store: %w", err)
}
s.store = store
var success bool
defer func() {
if !success {
_ = s.Close()
}
}()

Comment on lines +285 to +289
mux := http.NewServeMux()
cp.RegisterRoutes(mux)

// The console proxies /console/api/* to the loopback control plane and
// serves the embedded frontend; it queries /info at construction, which is

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Set the success flag to true on successful startup to prevent the deferred cleanup from rolling back the initialized server. The manual rtr.Close() call on failure can also be removed as it is now handled automatically by the deferred s.Close() call.

Suggested change
mux := http.NewServeMux()
cp.RegisterRoutes(mux)
// The console proxies /console/api/* to the loopback control plane and
// serves the embedded frontend; it queries /info at construction, which is
if s.publicAddr, err = s.resolvePublicAddr(); err != nil {
return err
}
success = true
return nil

Comment on lines +97 to +99
OIDCIssuer string
AllowedAudiences []string
// OIDCClientID is the OAuth client id advertised via /info.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Exposing ConnsPerSourceIP in standalone.Options allows operators to customize the per-source-IP connection limit. While a high limit (like 4000) is necessary behind TLS-terminating proxies or NATs, deployments running directly on a VM without a proxy should be able to lower this limit to protect against single-IP denial of service (DoS) attacks.

	OIDCIssuer       string
	AllowedAudiences []string
	// ConnsPerSourceIP overrides libp2p's per-source-IP inbound connection
	// cap when > 0. Defaults to router.DefaultHighWaterMark (4000).
	ConnsPerSourceIP int
}

Comment on lines +117 to +120
BiscuitTTL time.Duration
// ManualEnrollment queues bootstrap enrollments for admin approval
// instead of auto-approving them.
ManualEnrollment bool

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Set the default value for ConnsPerSourceIP to router.DefaultHighWaterMark if it is not explicitly configured.

	if len(o.AllowedAudiences) == 0 {
		o.AllowedAudiences = []string{api.DefaultAudience}
	}
	if o.ConnsPerSourceIP <= 0 {
		o.ConnsPerSourceIP = router.DefaultHighWaterMark
	}
}

if err := s.ensureBootstrapToken(ctx, s.joinToken, api.RoleNode, joinTokenMaxUsages, joinTokenTTL, "sam-one join token"); err != nil {
return fmt.Errorf("failed to register join token: %w", err)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Pass the configured ConnsPerSourceIP option to the router configuration.

Suggested change
ConnsPerSourceIP: s.opts.ConnsPerSourceIP,

Comment thread cmd/sam-one/main.go
Comment on lines +50 to +52
oidcClientID string
allowedAudiencesFlag string
logLevel string

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Declare a variable to hold the value of the new --conns-per-source-ip CLI flag.

Suggested change
oidcClientID string
allowedAudiencesFlag string
logLevel string
allowedAudiencesFlag string
logLevel string
connsPerSourceIP int
)

Comment thread cmd/sam-one/main.go
Comment on lines +97 to +99
DBDriver: dbDriver,
DBDSN: dbDSN,
JoinToken: joinToken,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Pass the parsed connsPerSourceIP value to standalone.Options.

				OIDCIssuer:       oidcIssuer,
				AllowedAudiences: auds,
				ConnsPerSourceIP: connsPerSourceIP,
			})

Comment thread cmd/sam-one/main.go
Comment on lines +128 to +129
rootCmd.Flags().StringSliceVar(&p2pListen, "p2p-listen", nil, "Optional extra native libp2p listen multiaddrs")
rootCmd.Flags().StringVar(&dataDir, "data-dir", ".", "Directory for the database, router key and generated tokens")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Register the --conns-per-source-ip flag on the root command to allow operators to customize this limit.

Suggested change
rootCmd.Flags().StringSliceVar(&p2pListen, "p2p-listen", nil, "Optional extra native libp2p listen multiaddrs")
rootCmd.Flags().StringVar(&dataDir, "data-dir", ".", "Directory for the database, router key and generated tokens")
rootCmd.Flags().StringVar(&allowedAudiencesFlag, "allowed-audiences", api.DefaultAudience, "Comma-separated list of allowed OIDC audiences")
rootCmd.Flags().StringVar(&logLevel, "log-level", "", "Log level: debug, info, warn, error")
rootCmd.Flags().IntVar(&connsPerSourceIP, "conns-per-source-ip", 0, "Max inbound connections per source IP (0 keeps libp2p's default of 8); raise behind TLS-terminating proxies or NAT where many peers share source IPs")

@aojea
aojea merged commit de96818 into google:main Sep 6, 2026
22 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant