feat: east-west & egress identity-aware microsegmentation (service graph, dns.fqdn, pod labels)#420
Merged
Merged
Conversation
…ral + direction fields Adds the agent half of the east-west / lateral-movement detection chain. - Direction + behavior: populate ids.dst_home_net/dst_external_net, ids.src_pod_net/dst_pod_net and the flow.* behavioral group (unique_dst_ports, flows_per_min, dst_port_entropy, ...) at BOTH eval sites (kernel_pump XDP and the AF_PACKET fallback). POD_NETS/is_pod_ip distinguishes true pod-to-pod from node-to-node-over-public-IP. New BlockSource::Microseg/Behavioral. - Workload identity: IdentityMmdbWorker pulls identity.mmdb on the threat-MMDB rails (push-aware via config SSE + interval-poll fallback); a security::identity lookup module resolves src/dst IP -> workload/namespace, surfaced as id.* fields. - Declared-edge microsegmentation: security::edge_set consumes the policy-edges allow-list; edge.declared / edge.policy_violation evaluated per flow. - Per-rule `log` smart-firewall action (records a notice, installs no kernel block) for staged alert-first rollout. - Overlay VXLAN/Geneve decap in the IDS inspect path so pod-to-pod overlay frames decode. Requires the matching amygdala scheme fields (ids.dst_*, ids.*_pod_net, flow.*, id.*, edge.*, Action::Log). The amygdala dependency + Cargo.lock need bumping once that publishes; build is blocked until then. Claude-Session: https://claude.ai/code/session_01MAY62VuZHhkeXgQ9feF3fJ
…rary The IdentityInfo/IdentityClient model + MMDB reader and the EdgeSet parser/ evaluator move to the hippocampus crate; synapse-core keeps the process-global client singletons + capability registration as thin wrappers that re-export the library types. Public API (lookup/evaluate/init_*/refresh_*/get_version_cache, IdentityInfo/EdgeVerdict) is unchanged, so the workers and enrichment call sites are untouched. hippocampus is a dev path dependency for now; switch to the gen0sec registry once it is published.
hippocampus is published, so synapse-core depends on it via the registry instead of a local path. A commented [patch.gen0sec] entry keeps the local-path dev override available, matching the amygdala/cortex pattern.
amygdala 0.1.6 publishes the id./edge./flow./ids. wirefilter enrichment fields this branch depends on; synapse-app now builds against the registry crate.
Process-global, bounded, idle-TTL'd accumulator (mirrors flow_signals): records observed src->dst:port edges (flow_count) and per-IP passive fingerprints (server JA4TS, client JA4T, client software) on the capture path; drains the slice for the ServiceGraphWorker to resolve + upload. Identity is resolved at drain, never on the hot path.
…'s slice Drains the observed buffer each tick, resolves IPs to workloads via the identity MMDB (host/<ip> fallback on-prem), and PUTs two contributions to service-graph-api: a declared slice (classification from identity + declared edges folded from the policy-edges artifact) and an observed slice (this agent's edges + passive fingerprints). Pure, tested projection; on-prem degrades gracefully (no artifacts -> observed-only, host refs). Links hippocampus with the graph feature (synapse-core Cargo.toml).
- Record observed east-west edges (both endpoints in HOME_NET) + passive fingerprints (client JA4T->src, server JA4TS->dst) at BOTH capture sites (kernel_pump XDP + AF_PACKET fallback), beside flow_signals::record. - platform.service_graph upload config (enabled/url/interval_secs). - Register the ServiceGraphWorker when configured (off in PCAP replay).
…ation) 0.0.1 lacked the graph feature + classification fields this worker needs. CI is green once hippocampus 0.0.2 publishes (gen0sec/hippocampus#8); dev builds via the local [patch] (uncommitted).
The agent owns only the observed layer and has no Kubernetes identity (it must run on-prem), so the service-graph worker now drains its observed edges + passive fingerprints and uploads a single `observed` slice keyed by raw IP. service-graph-api resolves those IPs to workload refs server-side against the operator's declared IP index (host/<ip> when unresolved), so the agent no longer folds identity/policy-edges itself. Drops the declared upload, identity-MMDB resolution and policy-edges fold from the worker; ServiceGraphConfig loses policy_edges_path.
The SYN-ACK (ja4ts) collector emits ServerToClient events with the server (this host) as src, so the previous src=client / dst=server mapping recorded the server fingerprint against the client and gated it on dst_home — leaving ja4ts empty in practice while client ja4t worked. Normalize each observation to client->server by PacketDirection: ja4t -> client, ja4ts -> server, edge client->server:server_port. Draw edges only from real flows (synthetic fingerprint carriers have src_port 0). Applied at both the XDP (kernel_pump) and AF_PACKET (lib.rs) capture sites.
On overlay clusters the egress SYN-ACK map is empty — the server's SYN-ACK is handled by the CNI datapath and never traverses the physical-NIC TC egress hook, so iter_egress() yields nothing while iter_egress_syn() (host-originated SYNs) works. The same outbound connection's SYN-ACK does arrive at ingress, so look the server's JA4TS up in the ingress SYN-ACK map (keyed by the connection's dst ip/port) and attach it to the egress-SYN event. That event is client->server, so the service graph attributes the fingerprint to the server node.
The TC-based SynAckCollector sees no SYN-ACKs on overlay/Cilium clusters
(the CNI's TC datapath owns the reply path), so server JA4TS was always
empty there. XDP runs at the driver, before the CNI's TC programs, so
inbound SYN-ACKs are visible.
Add an `xdp_synack_capture` program (separate __noinline, observe-only, no
blocking) that extracts the SYN-ACK fingerprint and stores it in new
server-keyed simple maps (tcp_synack_simple{,_v6}). Implement the
previously-stubbed `lookup_synack_fingerprint`/`_force` to read those maps,
which also revives the proxy's upstream-JA4TS lookup. Wire the service
graph's server-fingerprint recording to fall back to this lookup, and
attribute it to the server node by flow direction.
The per-event JA4TS lookup only fires when a flow event coincides with a captured SYN-ACK, so it misses periodic short-lived handshakes (the SYN-ACK lands in the map a beat after the egress-SYN event is processed). Add `drain_synack` (batch-iterate the server-keyed synack maps) and call it on a throttled cadence (every 30s) from the kernel pump, recording JA4TS for every internal server the node has handshaked with since startup — independent of flow-event timing.
On an overlay (Cilium VXLAN) cluster the physical NIC only sees encapsulated node-level traffic, so the observed graph is node-to-node. Add `capture.cni_interface` — a passive AF_PACKET SYN-only tap on the CNI tunnel netdev (e.g. cilium_vxlan), where decapsulated inner pod packets are visible. It never attaches XDP/TC (can't clobber the CNI datapath) and runs alongside the primary XDP capture; copies feed the same enrich pipeline so pod-to-pod edges land in the service graph and resolve to pod workloads via the operator's declared pod IPs. SYN-only keeps volume low; internal pod IPs are HOME_NET so they're never banned. Also fix the service-graph worker agent_id default to fall back to HOSTNAME (not a shared "synapse" literal) so each agent owns a distinct OBSERVED_EDGE slice instead of resync-clobbering the others.
Adds the production path for east-west pod-to-pod visibility on overlay (Cilium VXLAN) clusters: a tc/sched_cls program (tcx_overlay.bpf.c) attached via tcx ingress on the CNI tunnel netdev (e.g. cilium_vxlan) at BPF_TCX_FIRST — ahead of the CNI's own tcx program, so it sees the decapsulated inner pod packets first. Observer-only (always TC_ACT_UNSPEC), so it coexists with Cilium's tcx programs and never touches the datapath. It emits SYN/SYN-ACK connection-setup packets (raw TCP options included) over a ringbuf; the userspace consumer records pod-to-pod edges + JA4T/JA4TS into the service-graph buffer, resolved to pod workloads server-side via the operator's declared pod IPs (pod CIDR is HOME_NET, so never banned). `capture.cni_mode` selects the backend: `tcx` (default, this) or `af_packet` (the portable passive tap). Only active when `capture.cni_interface` is set.
Wires hippocampus's scan matcher into the agent so graph nodes get a service/product label, not just fingerprints. The tcx overlay program now also captures each flow's FIRST payload packet (the banner) — deduped via an LRU map, up to 256 bytes via bpf_skb_load_bytes — alongside the SYN/SYN-ACK connection events, over the same ringbuf (kind discriminator). The userspace consumer runs ProbeDb::identify(NULL/GetRequest/GenericLines, banner); a match labels the sender (the server) with service + "product version" via record_node_service. synapse-core links hippocampus with the scan feature (pcre2, vendored); the DB (nmap-service-probes, NPSL/GPL) is loaded from /var/lib/synapse at startup and is external data, never bundled in the crate.
…tes ladder (verifier)
…hadows /var/lib/synapse)
…er), not client requests
A workload can listen on several ports (postgres:5432 + a metrics http port). Key detected services by server port instead of collapsing to one, so they no longer overwrite each other. The buffer keeps a port->service map; drain emits the full per-port list plus the lowest-port primary for the node label. The passive matcher and active prober pass the server port through; the observed payload carries a services[] array per node.
The tcx tap recorded a flow only at SYN/SYN-ACK, so long-lived connections whose handshake we never saw (DB streaming replication, pooled channels) were invisible as edges. Add a kind=2 'established flow' event: any non-SYN packet emits the 5-tuple, deduped per-flow to once per 60s window via a timestamp LRU map (seen_flow). Userspace records the canonical client->server edge (lower port = server), no fingerprint. Proven live: database/core-1/2 -> core-0 streaming replication and the kafka inter-broker mesh now surface.
The overlay tap (cilium_vxlan) only sees east-west; pod egress to the internet (e.g. download-api -> S3) goes out the node's primary NIC, masqueraded, so it was invisible. Add a second tcx EGRESS program on the primary NIC at BPF_TCX_FIRST (before Cilium's BPF masquerade, so the pod IP is intact), filtered to pod-CIDR sources. Same event stream; the edge gate is relaxed to require only the client to be a pod, so external destinations are recorded. Config: capture.cni_egress_interface (name | 'auto' = default-route dev). External peers resolve to host/<ip> server-side and external/<reverse-dns> in the exporter (teal nodes). Best-effort: egress attach failure doesn't disable the east-west tap.
copy_raw_tcp_options{,_short} copied TCP options with a break-terminated,
variable-index byte loop. On stricter (newer-kernel) verifiers the index
becomes a runtime value and the verifier over-approximates the write to the
buffer end, rejecting the program load ("R3 min value is outside of the
allowed memory range" / "invalid write to stack") — which prevents the
XDP program from loading at all.
Replace the loop with a constant-size copy ladder: pick the largest fixed
block that fits within the packet bounds and __builtin_memcpy it, using a
constant length in each 'src + N <= data_end' check so the verifier can
prove the access. opt_total is always a multiple of 4, so out_len stays
exact. Mirrors the constant-size copy idiom already used for banner capture.
…in resilience Agent-side east-west security work (consumes the hippocampus identity/edge readers and the amygdala identity/edge scheme via workspace deps): - Identity/edge delta-stream apply: identity::apply_delta / edge_set::apply_delta wrappers + config_push SSE arms (identity-delta / policy-edges-delta) with an epoch/seq gap -> baseline resync. - Overlay-CNI identity microsegmentation: banned_edges BPF LRU_HASH + TC_ACT_SHOT drop in the tcx overlay; ban_edge/unban_edge; kernel_pump diverts a Drop verdict on a resolved pod-to-pod edge to an edge-drop (not an IP ban, since the pod IP is never-ban); the overlay consumer evaluates the observed edge so enforcement works on overlay CNIs where kernel_pump never sees the decapsulated flow. - Verifier-safe TCP options copy in the tcx overlay (constant-index write) for the stricter newer-kernel verifier. - Resilient BPF pinning: probe bpffs writability and load unpinned when pin creation is denied, instead of failing the whole skeleton load. - TLS SNI bridge: sni_store carries the ClientHello SNI into the smart-firewall eval (amygdala tls.sni) for domain-based rules.
Snoop DNS responses on the overlay to map a connection's destination IP back to the queried hostname, so smart-firewall rules can filter egress by domain: `dns.fqdn matches "(^|\.)evil\.com$" -> block`, optionally scoped by workload identity (`... and identity.k8s.src_namespace eq "..."`). - tcx overlay: capture UDP/53 DNS *responses* (new kind=3 event) alongside the TCP conn/banner/flow events; a direct bounded constant-index copy of the payload (not a fixed-step load ladder, which truncates odd-sized answers). - fqdn_store: a self-contained DNS response parser (header, question name with compression pointers, A/AAAA answers + TTL) feeding a TTL-bounded IP -> hostname map keyed by the resolved address. - overlay consumer: parse each snooped response, record every A/AAAA answer. - eval: kernel_pump + the overlay microseg eval populate amygdala's dns.fqdn extra from fqdn_store.lookup(dst_ip); the overlay eval now also fires for pod->external edges (identities optional) so a Drop bans the flow. - tests/dns_fqdn_repro.rs: exercises the compile_and_install + evaluate path. Reliable regardless of the app TLS library (unlike SNI), matches before any handshake. Requires the tcx egress tap; the DNS snoop is cross-node (same-node pod<->CoreDNS bypasses the overlay). Depends on amygdala dns.fqdn.
Populate amygdala's src/dst label map fields from IdentityInfo.labels in the overlay microseg eval, so NetworkPolicy-style label selectors work: `identity.k8s.src_label["role"] eq "database"`, incl. fused with dns.fqdn / namespace. The identity delta-wire parse carries the labels map through to the in-memory overlay. Depends on hippocampus IdentityInfo.labels + amygdala 0.1.9.
… interfaces The east-west/DNS observer attaches to any netdev (overlay on k8s, primary NIC on a plain host/VM) — the cni_ naming was misleadingly Kubernetes-specific. Replace (pre-release, no back-compat aliases): cni_interface -> observe_interfaces (now a list) cni_egress_interface -> observe_egress_interfaces (now a list) cni_mode -> observe_mode start_overlay_tcx takes interface slices and loops the tcx attach (ingress + egress) into one ringbuf, so a multi-homed host or multiple overlays are all observed. Requires the deployed config to use the new field names.
The overlay/east-west eval site (dns.fqdn + identity.k8s.* rules) previously
acted only on Drop, so `log` rules were a silent no-op and `allow` rules were
an unaudited whitelist. Mirror the north-south kernel_pump path: match all three
amygdala actions — drop installs the kernel edge-drop, log records an
observation without enforcing, allow is an explicit whitelist exception — and
emit one block-log audit record (Block / Notice / Allow) per matched edge.
The action already flows end to end from remote config: WirefilterRule.action
-> reload tuple -> compile_and_install -> amygdala Action::{Allow,Log,Drop}.
inspect_packet takes the full 9-input per-packet inspection context and its trace path uses serde_json::json! (which expands to internal unwrap()s). Both trip clippy only under the bpf feature, so -p synapse-idp missed them but the --workspace wellness gate does not. Add the modules existing #[allow(clippy::too_many_arguments, clippy::disallowed_methods)] convention.
Both crates are now published to the gen0sec registry with the east-west surface:
- amygdala 0.1.9: identity.k8s.* rename, dns.fqdn, identity.k8s.{src,dst}_label maps
- hippocampus 0.0.3: delta-stream identity/edge client + IdentityInfo.labels
Bump the version requirements to the published versions and resolve them from the
registry; the dev-only [patch.gen0sec] path overrides stay commented out.
The rebase conflict resolution kept main's classifier gate (is_enabled + is_protected_ip HOME_NET skip) but the merged body calls the shared classify_and_record_traffic helper, so the cortex inference/TrafficContext imports were unused under the classifier feature.
pigri
force-pushed
the
feat/east-west-identity-microseg
branch
from
1608ab6 to
8f971e2
Compare
… docs CI on non-default targets caught three gating gaps in the east-west code (local wellness runs linux + bpf, so they were invisible): - firewall_noop.rs (the non-BPF firewall module selected when bpf is off) lacked overlay_capture, so kernel_pumps AF_PACKET path and the microseg eval site failed to resolve firewall::overlay_capture::ban_edge on the legacy/musl build. Add no-op ban_edge/unban_edge stubs mirroring the real modules not(bpf) path. - last_synack_drain is read only under linux+bpf (the XDP synack drain), so it tripped dead_code on Windows. Gate the lint with cfg_attr. - overlay_capture doc used [`ban_edge`]/[`unban_edge`] intra-doc links that did not resolve under cargo doc -D warnings; use plain code spans.
ServiceGraphWorker lives in another crate, so the intra-doc link was unresolved under cargo doc --workspace --no-deps -D warnings. Use a plain code span. Verified the full workspace doc build is clean this time (not just one crate).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
East-west & egress visibility + identity-aware microsegmentation for the agent. A single smart-firewall rule can now match on destination domain, source/destination workload identity (incl. pod-label selectors), and behavioural signals, and deny per-edge in-kernel. Works on Kubernetes (overlay CNI) and on a plain host/VM (same eBPF, different attach interface).
Depends on the published
amygdala 0.1.9+hippocampus 0.0.3(gen0sec registry).Service graph — east-west & north-south visibility
BPF_TCX_FIRSTas a pure observer (TC_ACT_UNSPEC).ServiceGraphWorkerassembles the agent's observed slice (edges + node fingerprints) and uploads it.Workload-identity microsegmentation
hippocampus(IP → namespace / workload / labels), delivered as an MMDB baseline + incremental delta stream (DashMap client,apply_delta/load_baseline, epoch/seq resync).identity.k8s.src_namespace/src_workloadand pod-label selectorsidentity.k8s.{src,dst}_label["key"].banned_edgesBPF map (TC_ACT_SHOT, TTL'd) — observer-first, drops opt-in per(src → dst).Egress-by-domain
dns.fqdnrules: snoop DNS responses (UDP/53) into an IP→domain store, match connections by resolved IP. Protocol-agnostic (immune to ECH, unlike SNI).Rule actions
dropinstalls the edge-drop,logrecords an observation without enforcing,allowis an explicit whitelist exception — each emits one block-log audit record. Mirrors the north-southkernel_pumppath.Config
cni_* → observe_*rename (observe_interfaces/observe_egress_interfaces/observe_mode), now interface lists — the observer attaches to any netdev (overlay on k8s,eth0on a VM), not Kubernetes-specific.Deps
amygdala 0.1.9(identity.k8s.* rename,dns.fqdn, pod-label maps),hippocampus 0.0.3(delta-stream client +IdentityInfo.labels), resolved from the registry.Wellness:
fmt --check,clippy --workspace --all-targets(default +classifier)--deny warnings, andcargo test --workspace --liball pass.