sam-one - #360
Conversation
Init bootstraps the signing keyring, discovers OIDC providers and starts the key-rotation loop; RegisterRoutes registers every handler on a caller owned mux. Start composes them, so cmd/sam-control-plane is unchanged. A single-binary distribution embedding the control plane calls Init + RegisterRoutes and never binds the control plane's own listener. Pinned by TestInitRegisterRoutesEmbedded, including empty-issuer boot and Close without Start.
Options.HTTPFallbackHandler, when set, serves ordinary HTTP requests arriving on the router's /ws listen addrs so one public TCP port carries both libp2p and REST traffic. The transport list is assembled explicitly in that mode because combining DefaultTransports with an extra Transport(websocket.New, ...) registers the WS transport twice and fails host construction. TestSinglePortWebSocketAndHTTP pins the whole topology: an OIDC-less auto-approve control plane, a router self-enrolled over loopback with a store-seeded bootstrap token, and control-plane routes plus a libp2p dial both served through the router-owned port.
internal/standalone wires the shared SQL store, the control plane on a loopback-only listener, and the router owning the public port: WebSocket upgrades become libp2p connections while other HTTP requests fall through to the control-plane mux. The embedded router keeps its stock enrollment, lease and keys-sync flows against loopback, authenticated by a per-boot single-use bootstrap token, so no component grows in-process shortcuts. First boot provisions everything a zero-config mesh needs: signing keyring, persisted admin and join tokens (0600 files in --data-dir), and a mesh policy seed, either from --policy-file (protojson, first boot only, DB authoritative afterwards) or an explicitly-logged open development default. --external-url derives the advertised ws/wss multiaddr for platforms that terminate TLS at the edge (Cloud Run). cmd/sam-one is the cobra entrypoint with SAM_TOKEN / SAM_ADMIN_TOKEN / SAM_EXTERNAL_URL env fallbacks and a quickstart banner. TestStandaloneNodeJoin pins the CUJ: first boot, then a stock sam-node enrolls with the generated join token and connects to the embedded router over WebSocket through the single public port, in under a second.
Assets move to internal/console/public so go:embed works in-package; the console serves them from the embedded FS by default while --static-dir keeps overriding from disk for the ui-dev live-edit workflow. sam-one mounts the console at /console on the shared single-port mux, constructed after the listener is live so its /info discovery succeeds over loopback.
token create/list and admin ban are thin clients of the running server's /admin endpoints authenticated with the admin token (flag, SAM_ADMIN_TOKEN, or the token persisted in --data-dir), so a second process never opens the SQLite file. /admin/bootstrap-tokens gains an admin-gated GET returning the stored records to back token list.
Dockerfile.sam-one follows the distroless nonroot + /data-seed pattern and pins a stable container port via CMD; goreleaser gains the sam-one build. --bind-address (host) and --port are now independent flags with the port defaulting to 0: a free port is picked and published in the startup banner like the generated tokens, instead of baking any platform's port contract into the code. tests/e2e/standalone.bats boots the real binary on a random port, parses the banner, joins two real nodes, and pushes a request from node B through its egress proxy across the router to a smoke HTTP service registered on node A - pinning the single-port dataplane end to end, plus the token create/list CLI against the live server.
Behind a TLS-terminating proxy or NAT (verified on Cloud Run) every peer and every fallback HTTP request shares a handful of source IPs, and three stacked per-IP defaults in libp2p's resource manager each take the whole single-port listener down under normal reconnect churn: the inbound connection cap (8), the connection rate limit (0.2/s, burst 16 - the denials surface as 'rate limit exceeded' and self-heal as the bucket refills, which on Cloud Run reads as minutes of 429 'no available instance'), and the small transient scope that zombie conns from killed peers pool in. Options.ConnsPerSourceIP (sam-router: --conns-per-source-ip; 0 keeps the libp2p defaults) scales all three together: the per-subnet conn caps, the per-subnet rate limiter (same shape as upstream's, loopback stays exempt) and the system/transient scope floors. sam-one pins it to the connection manager's high watermark so the global cap is what binds. Root-caused with tcpdump (server SYN-ACKs, reads the request, then RSTs while the app and kernel counters look healthy) plus rcmgr debug logs; reproduced and regression-tested by churning relay-only nodes against a containerized sam-one: 132/160 failed probes before, 0/160 after.
Joining a mesh now always requires an explicit control plane (or the one already stored from a previous enrollment): the interactive public-mesh chooser is gone, 'sam-node join' without a URL errors out, 'run --join' demands --control-plane, and the mobile FFI refuses to start unenrolled without a configured ControlPlaneURL instead of pointing itself at the testnet. The testnets keep running; nodes just never end up on them by default.
Verified end to end: single-port wss enrollment, a relayed service call between two NAT-hidden nodes through the Cloud Run router, and the admin CLI against the public URL. Documents the load-bearing flags (singleton, no CPU throttling, request timeout), the reserved /healthz path, ephemeral state, and rollout overlap behavior.
Operator knobs forward to the components under --control-plane-* and --router-* prefixes (lease/rotation/grace/biscuit-ttl, manual enrollment, sync intervals, watermarks, per-source-IP budget, DHT record lifetimes, loopback advertisement). Structural wiring (loopback CP listener, shared store, derived ws listener) deliberately stays non-configurable; zero values keep each component's defaults.
There was a problem hiding this comment.
Code Review
This pull request introduces sam-one, an all-in-one standalone server distribution that combines the control plane, libp2p router, and storage into a single binary serving a single public port. It also embeds the web console assets directly into the binary and updates sam-node to require an explicit control plane URL when joining a mesh. The review feedback recommends implementing a deferred cleanup block in standalone.Start to prevent resource leaks on initialization failures, and exposing the ConnsPerSourceIP setting as a configurable option and CLI flag in sam-one to allow customization of connection limits behind proxies.
| func (o *Options) Validate() error { | ||
| if _, err := wsListenMultiaddr(o.BindAddress); err != nil { | ||
| return fmt.Errorf("invalid bind address %q: %w", o.BindAddress, err) | ||
| } | ||
| if o.ExternalURL != "" { |
There was a problem hiding this comment.
If Start fails after initializing the database store or the control plane, these resources (including background goroutines, HTTP servers, and database connections) are leaked if standalone is used as an embedded library. Adding a deferred cleanup block that calls s.Close() on failure ensures all partially allocated resources are cleanly released.
| func (o *Options) Validate() error { | |
| if _, err := wsListenMultiaddr(o.BindAddress); err != nil { | |
| return fmt.Errorf("invalid bind address %q: %w", o.BindAddress, err) | |
| } | |
| if o.ExternalURL != "" { | |
| store, err := storage.NewSQLStore(s.opts.DBDriver, s.opts.DBDSN) | |
| if err != nil { | |
| return fmt.Errorf("failed to open store: %w", err) | |
| } | |
| s.store = store | |
| var success bool | |
| defer func() { | |
| if !success { | |
| _ = s.Close() | |
| } | |
| }() |
| mux := http.NewServeMux() | ||
| cp.RegisterRoutes(mux) | ||
|
|
||
| // The console proxies /console/api/* to the loopback control plane and | ||
| // serves the embedded frontend; it queries /info at construction, which is |
There was a problem hiding this comment.
Set the success flag to true on successful startup to prevent the deferred cleanup from rolling back the initialized server. The manual rtr.Close() call on failure can also be removed as it is now handled automatically by the deferred s.Close() call.
| mux := http.NewServeMux() | |
| cp.RegisterRoutes(mux) | |
| // The console proxies /console/api/* to the loopback control plane and | |
| // serves the embedded frontend; it queries /info at construction, which is | |
| if s.publicAddr, err = s.resolvePublicAddr(); err != nil { | |
| return err | |
| } | |
| success = true | |
| return nil |
| OIDCIssuer string | ||
| AllowedAudiences []string | ||
| // OIDCClientID is the OAuth client id advertised via /info. |
There was a problem hiding this comment.
Exposing ConnsPerSourceIP in standalone.Options allows operators to customize the per-source-IP connection limit. While a high limit (like 4000) is necessary behind TLS-terminating proxies or NATs, deployments running directly on a VM without a proxy should be able to lower this limit to protect against single-IP denial of service (DoS) attacks.
OIDCIssuer string
AllowedAudiences []string
// ConnsPerSourceIP overrides libp2p's per-source-IP inbound connection
// cap when > 0. Defaults to router.DefaultHighWaterMark (4000).
ConnsPerSourceIP int
}| BiscuitTTL time.Duration | ||
| // ManualEnrollment queues bootstrap enrollments for admin approval | ||
| // instead of auto-approving them. | ||
| ManualEnrollment bool |
There was a problem hiding this comment.
| if err := s.ensureBootstrapToken(ctx, s.joinToken, api.RoleNode, joinTokenMaxUsages, joinTokenTTL, "sam-one join token"); err != nil { | ||
| return fmt.Errorf("failed to register join token: %w", err) | ||
| } | ||
|
|
| oidcClientID string | ||
| allowedAudiencesFlag string | ||
| logLevel string |
| DBDriver: dbDriver, | ||
| DBDSN: dbDSN, | ||
| JoinToken: joinToken, |
| rootCmd.Flags().StringSliceVar(&p2pListen, "p2p-listen", nil, "Optional extra native libp2p listen multiaddrs") | ||
| rootCmd.Flags().StringVar(&dataDir, "data-dir", ".", "Directory for the database, router key and generated tokens") |
There was a problem hiding this comment.
Register the --conns-per-source-ip flag on the root command to allow operators to customize this limit.
| rootCmd.Flags().StringSliceVar(&p2pListen, "p2p-listen", nil, "Optional extra native libp2p listen multiaddrs") | |
| rootCmd.Flags().StringVar(&dataDir, "data-dir", ".", "Directory for the database, router key and generated tokens") | |
| rootCmd.Flags().StringVar(&allowedAudiencesFlag, "allowed-audiences", api.DefaultAudience, "Comma-separated list of allowed OIDC audiences") | |
| rootCmd.Flags().StringVar(&logLevel, "log-level", "", "Log level: debug, info, warn, error") | |
| rootCmd.Flags().IntVar(&connsPerSourceIP, "conns-per-source-ip", 0, "Max inbound connections per source IP (0 keeps libp2p's default of 8); raise behind TLS-terminating proxies or NAT where many peers share source IPs") |
is the all-in-one SAM distribution: control plane, libp2p router
and storage in a single binary serving a single public port.