diff --git a/CHANGELOG.md b/CHANGELOG.md index 4273f62..a9beb07 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,122 @@ project adheres to [Semantic Versioning](https://semver.org/). ## [Unreleased] +### Added + +- A web UI, served by unified-api itself on the same address + (`server.ui.assets_dir`): sources, endpoints, projects, credentials, the + federation topology, metrics and the configuration, with a host search + across all of it. It is [unified-ui](https://github.com/OpusProjects/unified-ui) + v0.1.0, built into the image at `/usr/share/unified-api/ui`; each release + pins the unified-ui version it ships (`UI_REF` in `docker/Dockerfile`). An + unknown API path stays a JSON 404, and `/metrics` is the Metrics page only + for a browser asking for HTML. See `docs/ui.md`. +- LDAP login for the browsable UI (`server.ui.ldap` in `config.yaml`): people log + in with their directory account, and `access` rules map groups or usernames to + `admin` or to a restricted list of sources and endpoints, as `api_keys.yaml` + does. The service account resolves from `credentials.yaml` (env, file or + Vault). Plain `ldap://` without StartTLS, an unknown credential or an empty + rule list fail validation, and LDAP with no API key at all refuses to start + (or to reload): without a key the API runs open and the sign-in would + protect nothing. A reload that changes the block ends the LDAP sessions. API + key login keeps working beside it. See `docs/ldap-login.md`. +- `GET /api/v1/auth`: who the caller is and how they authenticated; for an admin, + also the LDAP configuration with its access rules and the API keys by name and + scope, never a secret. Linked from the API index as `auth`. +- Process metrics on `/metrics` under the standard Prometheus names: + `process_resident_memory_bytes`, `process_virtual_memory_bytes`, + `process_cpu_seconds_total`, `process_threads`, `process_open_fds` and + `process_max_fds`. Read from `/proc` at scrape time (Linux), with no new + dependency. The cache lives in memory, so resident memory is the number to + alert on before a pod meets its limit. +- `unified_api_hosts_unique` on `/metrics`: unique hostnames across every + cached source. Summing `unified_api_source_hosts` counts a host once per + source that lists it; this counts it once, ignoring case. +- `GET /api/v1/topology`: the federation seen from this instance. Its remote + sources grouped by the URL they read (one entry per peer instance, with each + source's cache and pull health), and each peer asked to describe itself in + turn, with the key those sources already use: a central's map shows its + edges. `?depth=` (default 2, at most 3) bounds the hops; a peer that will ask + further gets 8 seconds per level below it, so one dead instance two hops away + cannot hide a healthy one in between. Any key may ask and sees the peers + behind the sources it may read; a restricted key sees each peer's version + only. A peer that cannot be asked shows the reason; one older than this route + reports its version only. +- `?next=` on the login form: after logging in, the browser returns to the page + that sent it, as long as that page is on this origin. +- `server.ui.home`: where a login with no `?next=` lands (default `/` when + this instance serves the UI, `/api/v1/` otherwise). Logout takes the same + `next` and hands it to the login page, so signing out and back in from the + UI returns to the UI. +- `GET /api/v1/hosts/search?q=`: find a host by any part of its name across + every source and view the key may read, from the cache. Each host comes with + the sources (or views) holding it and the age of their data, and the + endpoints whose output includes it, by their sources, limit and own settings. + Best match first; `limit` (default 20, at most 100) and the total matched. +- `GET /api/v1/endpoints/{id}/counts`: hosts and groups in an endpoint's + default output, from the same merge and filters a call runs, without + rendering it. Null counts and `missing_sources` while a call would answer + 503; null for a script endpoint, and groups null for CSV. +- `GET /api/v1/credentials` (admin): each credential's type, where its secrets + are read from (env var names, file, Vault path), whether they resolve right + now, and what uses it (sources, projects, the LDAP login). Never a value. +- `GET /api/v1/settings` (admin): the configuration in force, defaults + included, grouped as `config.yaml` is: server, UI, cache snapshot, projects, + secrets and Vault (address, mount, how it signs in; never a token), the + config API, the running version and uptime, the snapshot's last save, and + any keys a reload left waiting for a restart. +- `GET /api/v1/endpoints`: an admin also gets each endpoint's `transform` + (builtin or script, project, arguments, timeout, limit, config key names). +- `GET /api/v1/projects`: `schedule`, `timeout_seconds` and `used_by` (the + sources, endpoints and enrichers whose `project_id` names the project). +- `GET /api/v1/sources/{id}`: an admin also gets `remote_url`, the instance a + remote source pulls from. +- The API index links `credentials` and `settings`. +- `server.ui.ldap.username_attribute` (default `sAMAccountName`): the + directory attribute holding an account's own name. Access rules' `users:` and + the session name use it rather than what was typed. +- LDAP login limits: after 5 wrong passwords in 15 minutes an account is + refused (429) without asking the directory, at most 4 directory logins run at + once, and every failed login answers after the same total time, so timing + does not reveal which usernames exist. +- `/metrics` with `server.metrics_require_auth` also accepts a signed-in + browser session, as every other route does. + +### Changed + +- **Breaking (anyone running the image on its bundled config):** the bundled + `config/config.yaml` now turns the UI on (`server.ui.enabled: true` and + `assets_dir`). `/` serves the UI instead of redirecting to `/swagger-ui/`, + and a browser opening an `/api/v1/` route gets the browsable HTML page. A + deployment with its own configuration is unaffected. + +### Fixed + +- The bundled config's sample project pointed at a repository that does not + exist, so every instance started on it logged a failed clone, and retried + it, every 30 minutes. It now clones unified-api itself, which holds the + sample scripts at the paths the sample sources name. +- The image could not clone the bundled config's projects: their default + directory (`projects`, under the root-owned `/app`) could not be created by + the unprivileged user the process runs as, so every sync that needed a + script failed with `Permission denied`. The image now creates it, owned by + that user. +- `process_cpu_seconds_total` counted only the main thread (from + `/proc/self/schedstat`), so it read close to zero while the runtime's worker + threads did the work. It now reads the whole process's user and system time + from `/proc/self/stat`. `process_open_fds` no longer counts the descriptor + used to count them. +- `server.ui.session_ttl_seconds` was documented as reloadable but is fixed + when the routes are built: a reload that changes it now reports it in + `restart_required`, as `server.ui.enabled` already was. + +- After a refused login (wrong key or password), the re-rendered form carried no + login challenge, so the corrected attempt was rejected as an expired form until + the page was reloaded. A refusal now issues a fresh challenge. +- A `UiConfig` built in code (rather than parsed) had a session TTL of zero, so + every session it issued was already expired. Parsed configuration was not + affected. + ## [0.33.0] - 2026-10-24 ### Added diff --git a/CLAUDE.md b/CLAUDE.md index e3f4ac0..6213431 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -55,7 +55,9 @@ src/ │ ├── static_inventory.rs # Ansible YAML inventory parsing (native, no process) │ ├── project.rs # GitProject │ ├── endpoint.rs # OutputEndpoint -│ └── view.rs # View, ViewMember, Ownership (read-only composite) +│ ├── view.rs # View, ViewMember, Ownership (read-only composite) +│ ├── ldap.rs # LdapConfig, AccessRule (LDAP sign-in) +│ └── topology.rs # Topology, Peer, PeerSource (federation map) ├── application/ # Use cases (domain + ports only; shared by HTTP and scheduler) │ ├── config.rs # reload: swap the RuntimeConfig, diff what changed │ ├── sync.rs # sync_source, SyncScope, SyncOutcome @@ -63,6 +65,9 @@ src/ │ ├── projects.rs # sync_project (git checkout up to date, bounded by timeout_seconds) │ ├── scripts.rs # per-execution script path resolution into project checkouts │ ├── views.rs # ViewSnapshot: owner resolution + merged reads +│ ├── login.rs # LDAP sign-in: directory lookup, access rules, throttle +│ ├── topology.rs # the federation map, asking peers recursively +│ ├── host_search.rs # find a host across sources and views from the cache │ └── credentials.rs # resolve_credentials ├── ports/ # Trait definitions (interfaces) │ ├── cache.rs # CachePort (incl. atomic update/merge_or_insert) @@ -72,6 +77,8 @@ src/ │ ├── git.rs # GitPort (project checkouts) │ ├── output.rs # OutputPort │ ├── secrets.rs # SecretsPort +│ ├── directory.rs # DirectoryPort (LDAP) +│ ├── peer.rs # PeerPort (another unified-api, for topology) │ └── venv.rs # VenvPort (per-project Python virtualenvs) ├── adapters/ # Everything that touches the outside world │ ├── in/ # Driving adapters: the outside world drives the app @@ -90,6 +97,8 @@ src/ │ │ │ ├── projects.rs # Git project routes (admin-only) │ │ │ ├── health.rs # /healthz, /readyz │ │ │ ├── metrics.rs # /metrics (Prometheus exporter, installed once) +│ │ │ ├── login.rs # login page, sessions, logout +│ │ │ ├── ui_assets.rs # the built web UI at "/" (server.ui.assets_dir) │ │ │ └── auth.rs # API key middleware │ │ └── scheduler/ # interval-based sync/enrich (calls application/) │ └── out/ # Driven adapters: the app drives the outside world @@ -100,6 +109,8 @@ src/ │ │ # remote.rs → another unified-api (federation) │ ├── enrichers/ # process.rs: EnricherPort → tokio::process │ ├── git/ # cli.rs: GitPort → git binary (clone/pull projects) +│ ├── directory/ # ldap.rs: DirectoryPort → ldap3 +│ ├── peers/ # http.rs: PeerPort → reqwest (topology) │ ├── output/ # process.rs: OutputPort → tokio::process │ └── secrets/ # env.rs: SecretsPort → env/JSON files; mock.rs: test double; │ # vault.rs; cache.rs; reloadable.rs (swappable chain); diff --git a/Cargo.lock b/Cargo.lock index fc5c0cd..e7fbb7b 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -113,6 +113,45 @@ version = "0.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7d902e3d592a523def97af8f317b08ce16b7ab854c1985a0c671e6f15cebc236" +[[package]] +name = "asn1-rs" +version = "0.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7f43a50ac4fdca5df8e885c21b835997f0a1cdee65494a6847694a98652d9d8" +dependencies = [ + "asn1-rs-derive", + "asn1-rs-impl", + "displaydoc", + "nom", + "num-traits", + "rusticata-macros", + "thiserror", + "time", +] + +[[package]] +name = "asn1-rs-derive" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3109e49b1e4909e9db6515a30c633684d68cdeaa252f215214cb4fa1a5bfee2c" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", + "synstructure", +] + +[[package]] +name = "asn1-rs-impl" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b18050c2cd6fe86c3a76584ef5e0baf286d038cda203eb6223df2cc413565f7" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + [[package]] name = "async-compression" version = "0.4.42" @@ -125,6 +164,17 @@ dependencies = [ "tokio", ] +[[package]] +name = "async-trait" +version = "0.1.92" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + [[package]] name = "atomic-waker" version = "1.1.2" @@ -477,6 +527,16 @@ version = "0.10.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c" +[[package]] +name = "core-foundation" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b2a6cd9ae233e7f62ba4e9353e81a88df7fc8a5987b8d445b4d90c879bd156f6" +dependencies = [ + "core-foundation-sys", + "libc", +] + [[package]] name = "core-foundation-sys" version = "0.8.7" @@ -752,6 +812,26 @@ dependencies = [ "zeroize", ] +[[package]] +name = "der-parser" +version = "10.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "07da5016415d5a3c4dd39b11ed26f915f52fc4e0dc197d87908bc916e51bc1a6" +dependencies = [ + "asn1-rs", + "displaydoc", + "nom", + "num-bigint 0.4.8", + "num-traits", + "rusticata-macros", +] + +[[package]] +name = "deranged" +version = "0.5.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e9de72ce2ad1f90dc62fa25f0f430ef85eb4b0d8fa0be4f30373bc40a21d28e" + [[package]] name = "derive_builder" version = "0.20.2" @@ -938,7 +1018,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" dependencies = [ "libc", - "windows-sys 0.52.0", + "windows-sys 0.61.2", ] [[package]] @@ -1318,6 +1398,12 @@ dependencies = [ "pin-project-lite", ] +[[package]] +name = "http-range-header" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9171a2ea8a68358193d15dd5d70c1c10a2afc3e7e4c5bc92bc9f025cebd7359c" + [[package]] name = "httparse" version = "1.10.1" @@ -1625,6 +1711,41 @@ version = "1.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe" +[[package]] +name = "lber" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cbcf559624bfd9fe8d488329a8959766335a43a9b8b2cdd6a2c379fca02909a5" +dependencies = [ + "bytes", + "nom", +] + +[[package]] +name = "ldap3" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "01fe89f5e7cfb7e4701e3a38ff9f00358e026a9aee940355d88ee9d81e5c7503" +dependencies = [ + "async-trait", + "bytes", + "futures", + "futures-util", + "lber", + "log", + "nom", + "percent-encoding", + "rustls", + "rustls-native-certs", + "thiserror", + "tokio", + "tokio-rustls", + "tokio-stream", + "tokio-util", + "url", + "x509-parser", +] + [[package]] name = "left-right" version = "0.11.7" @@ -1773,6 +1894,12 @@ dependencies = [ "unicase", ] +[[package]] +name = "minimal-lexical" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a" + [[package]] name = "miniz_oxide" version = "0.8.9" @@ -1843,6 +1970,16 @@ version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2bf50223579dc7cdcfb3bfcacf7069ff68243f8c363f62ffa99cf000a6b9c451" +[[package]] +name = "nom" +version = "7.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a" +dependencies = [ + "memchr", + "minimal-lexical", +] + [[package]] name = "nu-ansi-term" version = "0.50.3" @@ -1852,6 +1989,16 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "num-bigint" +version = "0.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c89e69e7e0f03bea5ef08013795c25018e101932225a656383bd384495ecc367" +dependencies = [ + "num-integer", + "num-traits", +] + [[package]] name = "num-bigint" version = "0.5.1" @@ -1864,6 +2011,12 @@ dependencies = [ "rand_core 0.10.1", ] +[[package]] +name = "num-conv" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441" + [[package]] name = "num-integer" version = "0.1.46" @@ -1882,6 +2035,15 @@ dependencies = [ "autocfg", ] +[[package]] +name = "oid-registry" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "12f40cff3dde1b6087cc5d5f5d4d65712f34016a03ed60e9c08dcc392736b5b7" +dependencies = [ + "asn1-rs", +] + [[package]] name = "once_cell" version = "1.21.4" @@ -1894,6 +2056,12 @@ version = "11.1.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d6790f58c7ff633d8771f42965289203411a5e5c68388703c06e14f24770b41e" +[[package]] +name = "openssl-probe" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe" + [[package]] name = "p256" version = "0.14.0" @@ -2148,6 +2316,12 @@ dependencies = [ "zerovec", ] +[[package]] +name = "powerfmt" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4a6394b9e965e73d0a289ee54f589087e2c676aedf60885baf52c76b771e4958" + [[package]] name = "ppv-lite86" version = "0.2.21" @@ -2261,7 +2435,7 @@ dependencies = [ "once_cell", "socket2", "tracing", - "windows-sys 0.52.0", + "windows-sys 0.61.2", ] [[package]] @@ -2544,7 +2718,7 @@ dependencies = [ "md5", "ml-kem", "module-lattice", - "num-bigint", + "num-bigint 0.5.1", "p256", "p384", "p521", @@ -2616,6 +2790,15 @@ dependencies = [ "semver", ] +[[package]] +name = "rusticata-macros" +version = "4.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "faf0c4a6ece9950b9abdb62b1cfcf2a68b3b67a10ba445b3bb85be2a293d0632" +dependencies = [ + "nom", +] + [[package]] name = "rustix" version = "1.1.4" @@ -2626,7 +2809,7 @@ dependencies = [ "errno", "libc", "linux-raw-sys", - "windows-sys 0.52.0", + "windows-sys 0.61.2", ] [[package]] @@ -2635,6 +2818,7 @@ version = "0.23.46" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "48e13bd8c0e9365c43cfa5c9e8f9ad49d3c8444926c9aac819e0e4dc503c8fdf" dependencies = [ + "log", "once_cell", "ring", "rustls-pki-types", @@ -2643,6 +2827,18 @@ dependencies = [ "zeroize", ] +[[package]] +name = "rustls-native-certs" +version = "0.8.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4a073f5dc7469f984c52ad2752b63b0807745133b6de880b7b64c1ac4c48aec4" +dependencies = [ + "openssl-probe", + "rustls-pki-types", + "schannel", + "security-framework", +] + [[package]] name = "rustls-pki-types" version = "1.15.0" @@ -2695,6 +2891,15 @@ dependencies = [ "winapi-util", ] +[[package]] +name = "schannel" +version = "0.1.30" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9cb23c424d8ee841bb4d813677ca237e2a610067d594fec130f87d6093495ab5" +dependencies = [ + "windows-sys 0.61.2", +] + [[package]] name = "scoped-tls" version = "1.0.1" @@ -2733,6 +2938,29 @@ dependencies = [ "zeroize", ] +[[package]] +name = "security-framework" +version = "3.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d" +dependencies = [ + "bitflags", + "core-foundation", + "core-foundation-sys", + "libc", + "security-framework-sys", +] + +[[package]] +name = "security-framework-sys" +version = "2.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ce2691df843ecc5d231c0b14ece2acc3efb62c0a398c7e1d875f3983ce020e3" +dependencies = [ + "core-foundation-sys", + "libc", +] + [[package]] name = "semver" version = "1.0.28" @@ -3113,10 +3341,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" dependencies = [ "fastrand", - "getrandom 0.3.4", + "getrandom 0.4.3", "once_cell", "rustix", - "windows-sys 0.52.0", + "windows-sys 0.61.2", ] [[package]] @@ -3148,6 +3376,36 @@ dependencies = [ "cfg-if", ] +[[package]] +name = "time" +version = "0.3.55" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cdb87b95ec50ddfa440816d227a17b2ccbdda963a316a727fda0fc4334f7d134" +dependencies = [ + "deranged", + "num-conv", + "powerfmt", + "serde_core", + "time-core", + "time-macros", +] + +[[package]] +name = "time-core" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e1c906769ad99c88eaa54e728060edef082f8e358ff32030cb7c7d315e81109" + +[[package]] +name = "time-macros" +version = "0.2.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e689342a48d2ea927c87ea50cabf8594854bf940e9310208848d680d668ed85" +dependencies = [ + "num-conv", + "time-core", +] + [[package]] name = "tinystr" version = "0.8.3" @@ -3221,6 +3479,17 @@ dependencies = [ "tokio", ] +[[package]] +name = "tokio-stream" +version = "0.1.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a3d06f0b082ba57c26b79407372e57cf2a1e28124f78e9479fe80322cf53420b" +dependencies = [ + "futures-core", + "pin-project-lite", + "tokio", +] + [[package]] name = "tokio-util" version = "0.7.18" @@ -3278,8 +3547,14 @@ dependencies = [ "bitflags", "bytes", "futures-core", + "futures-util", "http", "http-body", + "http-body-util", + "http-range-header", + "httpdate", + "mime", + "mime_guess", "percent-encoding", "pin-project-lite", "tokio", @@ -3428,10 +3703,13 @@ dependencies = [ "getrandom 0.4.3", "hmac", "http-body-util", + "ldap3", "metrics", "metrics-exporter-prometheus", "reqwest", "russh", + "rustls", + "rustls-pki-types", "serde", "serde-saphyr", "serde_json", @@ -3692,7 +3970,7 @@ version = "0.1.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" dependencies = [ - "windows-sys 0.52.0", + "windows-sys 0.61.2", ] [[package]] @@ -3917,6 +4195,23 @@ version = "0.6.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4" +[[package]] +name = "x509-parser" +version = "0.18.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d43b0f71ce057da06bc0851b23ee24f3f86190b07203dd8f567d0b706a185202" +dependencies = [ + "asn1-rs", + "data-encoding", + "der-parser", + "lazy_static", + "nom", + "oid-registry", + "rusticata-macros", + "thiserror", + "time", +] + [[package]] name = "yoke" version = "0.8.3" diff --git a/Cargo.toml b/Cargo.toml index 7b343f5..9b77d48 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -44,13 +44,22 @@ serde = { version = "1", features = ["derive", "rc"] } # rc: Serialize/Deseria serde_json = "1" serde-saphyr = "2" # pure-Rust YAML reader (no unsafe-libyaml); we only ever read YAML subtle = "2" +# LDAP login for the browsable API: bind, look the user up, bind as them. +# rustls on ring, the TLS stack and crypto provider already in the tree, so no +# OpenSSL comes with it. +ldap3 = { version = "0.12.1", default-features = false, features = ["tls-rustls-ring"] } +# A private CA for the directory (`ldap.ca_file`): the TLS config is built here +# and handed to ldap3. Both crates are already in the tree through ldap3 and +# reqwest. +rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"] } +rustls-pki-types = { version = "1", features = ["std"] } tokio = { version = "1", features = ["full"] } # The CORS middleware wraps tower-http's CorsLayer around the rest of the # stack per request (Layer + oneshot from `util`), which is what makes the # origin list reloadable. Already in the tree as an axum dependency — this # adds no new code to the build. tower = { version = "0.5", default-features = false, features = ["util"] } -tower-http = { version = "0.7", features = ["cors", "trace", "compression-gzip", "request-id"] } +tower-http = { version = "0.7", features = ["cors", "trace", "compression-gzip", "request-id", "fs"] } tracing = "0.1" tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] } utoipa = { version = "6", features = ["axum_extras"] } diff --git a/README.md b/README.md index 8e40c94..c9207a6 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,7 @@ and every one of them sees the same inventory. - **Scheduled + on-demand sync**: interval sync per source, plus scoped sync over the API - **Refresh**: a read can bring the hosts it names up to date, bounded by the source's TTL - **Configuration API**: a pipeline pushes the config directory and reloads it live, no restart +- **Web UI**: sources, endpoints, topology, metrics and host search in a browser, same address - **Swagger UI**: interactive OpenAPI docs served at `/swagger-ui/` - **Single static binary**: axum + tokio, hexagonal architecture, ~11k lines @@ -56,6 +57,7 @@ and every one of them sees the same inventory. | [Troubleshooting](docs/troubleshooting.md) | Symptom first: what to check when data is stale or a read refuses | | [TTL](docs/caching.md) | The three-level freshness model, sync modes and TTL overrides | | [Views](docs/views.md) | Several sources as one id, and how a host is routed to its owner | +| [Web UI](docs/ui.md) | The UI the image serves: turning it on, signing in, what is served where | --- diff --git a/config/config.yaml b/config/config.yaml index 25e2fed..a49c4c3 100644 --- a/config/config.yaml +++ b/config/config.yaml @@ -1,6 +1,13 @@ server: host: "0.0.0.0" port: 8182 + # The web UI on this same address: the image ships a build of unified-ui at + # /usr/share/unified-api/ui. Sign-in is the API's own login page (an API key, + # or LDAP: docs/ldap-login.md); with no api_keys.yaml the API is open and so + # is the UI. Remove the block for an API without a UI. See docs/ui.md. + ui: + enabled: true + assets_dir: "/usr/share/unified-api/ui" # Optional: snapshot the in-memory cache to disk so restarts start from the # last snapshot instead of empty. Remove the block for a purely in-memory cache. diff --git a/config/projects.yaml b/config/projects.yaml index a8d491e..9775974 100644 --- a/config/projects.yaml +++ b/config/projects.yaml @@ -1,6 +1,10 @@ +# The sample sources' scripts live in this repository (tests/adapters/out/), +# so the sample project is unified-api itself: public, so no credential, and +# the paths in sources.yaml resolve inside its checkout. A real deployment +# points this at its own connector repository. prj-connectors-infra: name: "Infrastructure Connectors" - git_url: "https://github.com/OpusProjects/connectors-infra.git" + git_url: "https://github.com/OpusProjects/unified-api.git" # branch: "main" # default "main" # credential_id: "cred-github-token" # for private repos (token or ssh_key) sync_interval_seconds: 1800 # re-pull every 30 min (0/absent = no periodic sync) diff --git a/docker/Dockerfile b/docker/Dockerfile index aff3a49..abcaaa8 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -1,4 +1,14 @@ -# Stage 1: build +# Stage 1: the web UI. unified-ui is its own repository with its own +# versions; each unified-api release pins the one it ships (UI_REF), so an +# image always serves the UI it was tested with. +FROM node:22-alpine AS ui +ARG UI_REF=v0.1.0 +RUN apk add --no-cache git \ + && git clone --depth 1 --branch "$UI_REF" https://github.com/OpusProjects/unified-ui.git /ui +WORKDIR /ui +RUN npm ci --no-audit --no-fund && npm run build + +# Stage 2: build FROM rust:1.96-slim-trixie AS builder RUN apt-get update && apt-get install -y pkg-config libssl-dev curl && rm -rf /var/lib/apt/lists/* @@ -20,7 +30,7 @@ RUN mkdir src benches \ COPY src/ src/ RUN touch src/main.rs && cargo build --release --locked -# Stage 2: runtime +# Stage 3: runtime FROM debian:trixie-slim # git: the app clones connector-script projects at boot (projects.yaml). @@ -38,11 +48,15 @@ RUN apt-get update && apt-get install -y \ RUN groupadd -r unified && useradd -r -g unified -s /sbin/nologin unified -# Writable state directory: project checkouts and the optional cache snapshot -# default here. In k8s, mount a volume over it to survive pod replacement. +# Writable state directory for what should survive a restart: the optional +# cache snapshot, and project checkouts when projects.dir points here (as +# docs/deployment.md does). In k8s, mount a volume over it. RUN mkdir -p /var/lib/unified-api && chown unified:unified /var/lib/unified-api COPY --from=builder /app/target/release/unified-api /usr/local/bin/unified-api +# Served at "/" when the config points server.ui.assets_dir here (the bundled +# config does). +COPY --from=ui /ui/apps/unified-api/dist /usr/share/unified-api/ui COPY config/ /app/config/ # Demo connector/enricher/output scripts the default config points at. They # double as the integration-test fixtures, so they live under tests/ (mirroring @@ -51,6 +65,12 @@ COPY tests/adapters/out/ /app/tests/adapters/out/ WORKDIR /app +# The bundled config clones its projects into the default `projects.dir`, +# relative to /app, as the unprivileged user below: /app itself stays +# root-owned (config and scripts are not the process's to change), so the +# checkout directory is created here and handed over. +RUN mkdir -p /app/projects && chown unified:unified /app/projects + USER unified EXPOSE 8182 diff --git a/docs/browsable-api.md b/docs/browsable-api.md index 165eca6..8a69de2 100644 --- a/docs/browsable-api.md +++ b/docs/browsable-api.md @@ -255,6 +255,10 @@ login form and a session cookie rather than a header box. If no keys are configured at all, the API is open (`auth.rs:213`) and so is the UI, with the same loud warning at startup. No special case. +People can also log in with a directory account instead of a key: see +[LDAP login](ldap-login.md). The session is the same; only the check at the +door differs. + ## Writes: forms and CSRF The point of the console is to be able to press **sync** — a read-only version diff --git a/docs/config-api.md b/docs/config-api.md index 1515848..7e161fd 100644 --- a/docs/config-api.md +++ b/docs/config-api.md @@ -166,7 +166,8 @@ is already running. **Needs a restart** — reported, never silently ignored: `server.host`, `server.port`, -`cache.persistence`, `projects.dir`, `config_api.enabled`. +`cache.persistence`, `projects.dir`, `config_api.enabled`, `server.ui.enabled`, +`server.ui.session_ttl_seconds`. A write that touches one of them still lands on disk; the response names it: diff --git a/docs/federation.md b/docs/federation.md index b650809..473bb74 100644 --- a/docs/federation.md +++ b/docs/federation.md @@ -207,6 +207,52 @@ Failure modes, all loud: --- +## Seeing the federation: `GET /api/v1/topology` + +Every instance can describe the federation from where it stands: + +```bash +curl -s -H "X-API-Key: $KEY" https://central.example.com/api/v1/topology?depth=2 | jq +``` + +```json +{ + "version": "0.34.0", + "sources": 30, + "peers": [ + { + "url": "https://unified-api-dc1.example.com:8443", + "sources": [ + { "source_id": "src-dc1", "remote_source": "src-fleet", "cached": true, + "hosts": 437, "age_seconds": 120, "consecutive_failures": 0 } + ], + "instance": { "version": "0.34.0", "sources": 3, "peers": [] } + } + ] +} +``` + +- **Peers come from configuration**: every `connector_type: remote` source, + grouped by its `config.url`. No registry to keep in sync. +- **Each peer is asked in turn**, with the token credential its sources + already use, and nests its own answer under `instance`. `depth` (default 2, + at most 3) bounds the hops, so a loop of instances federating each other + ends. +- **A peer that cannot be asked keeps its place** with `error` set + (`unreachable`, `did not answer in time`, a refused key). A peer older than + this route answers with its version only, and its own peers stay unknown. +- **Any key may ask; each sees the peers behind the sources it may read.** An + edge answers its central's restricted key, and nothing about sources outside + that key's scope. +- Each peer has 8 seconds to answer, plus 8 more for each level it will ask + below it: the map is a page someone is waiting on, not a sync, and a middle + peer still waiting on a dead one further away must not be mistaken for dead + itself. An older peer's fallback (its index) counts within the same time. +- A restricted key sees each peer's version, not the peer's own peers: those + were asked with this instance's key, not the caller's. + +The UI's Topology page draws this as a tree. + ## Operational notes How the arrangement behaves once it is running, and what it costs to change its shape. diff --git a/docs/ldap-login.md b/docs/ldap-login.md new file mode 100644 index 0000000..939a420 --- /dev/null +++ b/docs/ldap-login.md @@ -0,0 +1,172 @@ +# LDAP login + +The browsable UI's login form can take a directory account (LDAP or Active +Directory) as well as an API key. People log in with the username and password +they already have; what they may touch is decided by **access rules in this +service's configuration**, in the same vocabulary `api_keys.yaml` uses. + +The directory says who someone is and which groups they belong to. It never +decides access by itself: a new group in AD grants nothing until a rule names +it. + +## Configuration + +`config.yaml`, under `server.ui` (LDAP login is part of the browsable UI, so +`enabled: true` is required): + +```yaml +server: + ui: + enabled: true + home: "/" # where a sign-in lands; see "Sessions" + ldap: + url: "ldaps://ldap.example.com:636" # or ldap://...:389 with starttls: true + bind_credential: cred-ldap # credentials.yaml: username = bind DN, password + search_base: "OU=Users,DC=example,DC=com" + # user_filter: "(sAMAccountName={username})" # the default (Active Directory) + # group_attribute: memberOf # the default + # ca_file: /etc/ssl/private-ca.pem # a private CA; default = system store + # timeout_seconds: 10 + access: + - name: "Operations" + groups: [DL-pcol2operations] # a group CN, or a full DN + role: admin + - name: "VMware readers" + groups: ["CN=DL-vmware,OU=Groups,DC=example,DC=com"] + users: [jane.doe] # people without a group, by username + role: restricted # the default + sources: [src-vmware] + endpoints: [ep-ansible-vmware] +``` + +The service account comes from `credentials.yaml` like any connector +credential, so it can live in env vars, a file or Vault: + +```yaml +cred-ldap: + name: "LDAP service account" + type: username_password + vault_path: unified-api/ldap # or env_prefix / secret_file +``` + +Its `username` is the **bind DN** (`CN=svc-unified,OU=Service,DC=example,DC=com`) +and `password` its password. It only needs to read users and their group +attribute. + +### Access rules + +- Evaluated top to bottom, **all matches combined**: `admin` in any matching rule + is admin; otherwise the `sources` and `endpoints` of every matching rule are + unioned. +- A group written as a **CN** (`DL-ops`) matches any group with that CN; a **DN** + matches only that group. Both case-insensitive. +- `role` defaults to `restricted`, the same safe default as API keys: a rule that + forgets it never hands out admin. +- **Matching no rule refuses the login**, even with the right password. The person + is told their account has no access, which leaks nothing (they proved the + password) and saves a ticket. + +### Validation + +`--check-config` and the config API refuse, before anyone tries to log in: + +- `ldap://` without `starttls: true`, since every password would cross the + network in clear text; +- a `bind_credential` that is not in `credentials.yaml`; +- a `user_filter` without `{username}`; +- an empty `access` list, or a rule naming no groups and no users; +- a rule naming a source or endpoint that does not exist; +- `ldap:` with `server.ui.enabled: false`. + +## How a login works + +``` + form: username + password + │ + ├─ bind as the service account (bind_credential, via the secrets chain) + ├─ search search_base for user_filter ({username} escaped, RFC 4515) + │ exactly one entry, or the login fails + ├─ bind AS that entry with the password typed (the directory checks it) + ├─ read group_attribute (memberOf) + └─ access rules → admin | restricted(sources, endpoints) | refused +``` + +One connection per login, closed straight after. No password or hash is ever +stored or readable here. + +Guards worth knowing about: + +- **An empty password is refused before the directory is contacted.** To LDAP, a + DN with no password is an *unauthenticated bind*, which Active Directory and + many other servers answer with success. Without this guard an empty password + field would log anyone in as anyone. +- **The username is escaped** into the filter, so `*` or `)(` cannot rewrite it. +- **A filter matching more than one entry is refused** rather than picking the + first, and logged. +- **Wrong password and unknown user get the same answer**, after a fixed pause. + The directory's own lockout policy protects the accounts. +- **An unreachable directory is not a wrong password.** It answers `503`, logs + the reason, and suggests the API key, which keeps working. + +## Account names, limits, and the API keys it needs + +- **Whose rule applies.** `access[].users` and the session's name use the + account's own name, read from `username_attribute` (default + `sAMAccountName`), not what was typed. With a filter that matches on several + attributes, an e-mail address say, typing someone else's name cannot reach + their rule. +- **Limits.** After 5 wrong passwords in 15 minutes an account is refused (429) + without the directory being asked, so a script cannot lock the real account + out through this form; at most 4 directory logins run at once; and every + failed login answers after the same 1.5 seconds, so timing does not tell an + unknown user from a wrong password. +- **At least one API key.** With no API key configured the API runs open: every + caller is admin and no session is consulted. LDAP in that state would protect + nothing, so it refuses to start, and a reload that would arrive at it is + refused. An admin key is also the way in when the directory is down. + +## Sessions + +An LDAP login opens the same kind of browser session an API key does (in +memory, `HttpOnly`, `SameSite=Strict`, CSRF on every write). It is named +`ldap:` in the access log. + +After signing in, the browser goes back to the page that sent it there +(`?next=`, only a path on this server). With no such page (someone opening +`/api/v1/login` directly) it goes to `server.ui.home`: `/api/v1/` by default, +the API's own pages, and `"/"` where the UI is served in front of the API. +Signing out from the UI posts `next=/`, so signing straight back in returns to +the UI too. + +What ends it: + +- logout, or `session_ttl_seconds`; +- a restart; +- **a reload that changes `server.ui.ldap`**. A session carries the permissions + its rules granted at login, so changed rules end the LDAP sessions and the next + login is judged by the new ones. API key sessions are untouched (a key's + removal revokes those), and a reload that changes only sources leaves LDAP + sessions alone. + +`server.ui.ldap` and `server.ui.home` reload live (`applied` in the reload +report). `server.ui.enabled` and `server.ui.session_ttl_seconds` need a restart: +a reload that changes them reports them in `restart_required`. + +## Who is logged in: `GET /api/v1/auth` + +Read-only, for the UI's Users page. Every authenticated caller gets `me` (name, +method `ldap` / `api_key` / `open`, role, scope). An admin also gets the LDAP +configuration with its access rules, and the API keys by name and scope. A +secret never appears. Users and the directory are declared in git and pushed +like the rest of the configuration, so this route describes what is in force +rather than offering a second place to change it. + +## What it does not do + +- **No local users.** People log in through the directory; machines and + break-glass access use API keys. +- **No nested groups** yet: only the groups listed directly in `memberOf`. Active + Directory's transitive match (`1.2.840.113556.1.4.1941`) can be added as an + option if a rule needs it. +- **No editing from the UI.** Access is changed by a pull request to the + configuration repository. diff --git a/docs/observability.md b/docs/observability.md index 192abcf..696eade 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -131,6 +131,7 @@ probes — scrapers don't carry the API key): | `unified_api_source_fresh` | `source` | 1 while the dataset is within its TTL, 0 once expired — and 0 for a source that has only ever received host- or group-scoped syncs, which have no full gather for the TTL to be measured against | | `unified_api_source_hosts` | `source` | Hosts in the cached dataset | | `unified_api_source_groups` | `source` | Groups in the cached dataset | +| `unified_api_hosts_unique` | — | Unique hostnames across every cached source. Summing `unified_api_source_hosts` counts a host once per source that lists it; this counts it once. Case-insensitive (`AASQL101` = `aasql101`), but nothing else is folded: `web01` and `web01.example.com` are two, so a source that lists VM names or a second domain inflates it | | `unified_api_source_sync_consecutive_failures` | `source` | Failed syncs since the last success — 0 while healthy. Appears once the source has attempted a sync in this process | | `unified_api_source_sync_last_attempt_age_seconds` | `source` | Seconds since a sync was last **attempted**, successful or not. Growing past the sync interval = the scheduler task is not running at all | | `unified_api_source_sync_last_success_age_seconds` | `source` | Seconds since the last successful sync. Absent until one has succeeded | @@ -149,6 +150,11 @@ probes — scrapers don't carry the API key): | `unified_api_snapshot_consecutive_failures` | — | Failed cache snapshot writes since one last succeeded (a full disk, revoked permissions, a vanished volume) | | `unified_api_snapshot_last_success_age_seconds` | — | Seconds since a snapshot was last written | | `unified_api_scheduler_task_panics_total` | `task` | Panics caught and restarted by the task supervisor (`sync:`, `enrich:`, `project:`). Any non-zero value is a bug worth reporting | +| `process_resident_memory_bytes` | — | Resident memory. The whole inventory cache lives in it, so this is the number that says a pod is approaching its memory limit. Linux only, read from `/proc` at scrape time, like the `process_*` rows below | +| `process_virtual_memory_bytes` | — | Virtual memory size | +| `process_cpu_seconds_total` | — | User plus system CPU time of every thread since the process started (`/proc/self/stat`); `rate()` gives cores in use. Exported as a gauge (the exporter's counters are whole numbers), which `rate()` treats the same | +| `process_threads` | — | OS threads | +| `process_open_fds` / `process_max_fds` | — | File descriptors open, and the soft limit. Absent when the limit is `unlimited` | A view holds no cache entry of its own, so it has none of the `unified_api_source_*` series — its members do. The names are separate rather than reusing the source diff --git a/docs/ui.md b/docs/ui.md new file mode 100644 index 0000000..2eaa46f --- /dev/null +++ b/docs/ui.md @@ -0,0 +1,68 @@ +# Web UI + +The image serves a web UI on the same address as the API: open the instance's +address in a browser, sign in, and see its sources, endpoints, projects, +credentials, federation topology, metrics and configuration, with a host +search across all of it. + +The UI is its own project, [unified-ui](https://github.com/OpusProjects/unified-ui), +with its own versions. Each unified-api release builds the unified-ui version +it was tested with into its image (`UI_REF` in `docker/Dockerfile`), and serves +the built files itself: no proxy in front, no second container. + +## Turning it on + +```yaml +# config.yaml +server: + ui: + enabled: true + assets_dir: "/usr/share/unified-api/ui" # where the image puts the build +``` + +The bundled `config/config.yaml` already has this block, so the image serves +the UI out of the box. A deployment with its own configuration adds it. + +- `assets_dir` needs `enabled: true`: the UI signs in through the API's login + page and its session cookie, which exist only with it. Validation refuses + the one without the other. +- `assets_dir` is restart-only: the routes are built once at startup. A + reload that changes it says so in `restart_required`. +- A directory with no `index.html` is logged as a warning at startup; the API + works as before and only the UI is missing. + +## What is served where + +| Path | With a UI | Without | +|---|---|---| +| `/` | the UI | redirect to `/swagger-ui/` | +| `/sources`, `/topology`, … (any page of the UI) | the UI | 404 | +| `/assets/*` | the UI's files, cached for good (their names change with their content) | 404 | +| `/metrics` | the UI's Metrics page for a browser (`Accept: text/html`); the Prometheus text for everyone else | the Prometheus text | +| `/api/...`, `/swagger-ui/`, `/api-docs/` | unchanged; an unknown API path is still a JSON 404, never the UI | unchanged | + +The UI's own files are public: they hold no data. Every call the UI makes for +data goes through the same authentication as any other client. + +## Signing in + +The UI has no login of its own. A call without a session gets a 401, and the +UI sends the browser to the API's login page (`/api/v1/login`, an API key or +LDAP: see [ldap-login.md](ldap-login.md)), with `?next=` set to the page it +was on. After signing in the browser returns there. + +A login with no `next` (someone opening `/api/v1/login` directly) lands on +`server.ui.home`, which defaults to `/` when `assets_dir` is set and to the +API's own pages (`/api/v1/`) when it is not. Signing out from the UI returns to +the login page with `next=/`, so signing straight back in returns to the UI. + +What a person can see and do is what their key, or their LDAP access rule, +allows: a restricted key sees only its sources and endpoints, and the +admin-only pages (projects, credentials, configuration) are left out of the +menu. + +## Running the UI from its own repository + +For working on the UI, unified-ui runs it on Vite against any unified-api +(`npm run dev`), and has a demo image with nginx in front of the API. Both are +described in its README; neither is needed to use the UI. diff --git a/src/adapters/in/http/auth.rs b/src/adapters/in/http/auth.rs index 1c803ae..db6a042 100644 --- a/src/adapters/in/http/auth.rs +++ b/src/adapters/in/http/auth.rs @@ -146,6 +146,27 @@ pub fn resolve_api_keys(cfg: &crate::config::AppConfig) -> Result Result<(), String> { + if cfg.server.ui.ldap.is_some() && keys.is_empty() { + return Err( + "server.ui.ldap is configured but no API key is: without one \ + the API runs with no authentication at all, and the LDAP sign-in \ + would protect nothing. Declare at least one key in api_keys.yaml \ + (an admin key also stays the way in when the directory is down)" + .to_string(), + ); + } + Ok(()) +} + // A browser session's identity, published on the response for the HTML layer. #[derive(Clone)] pub struct BrowserIdentity { @@ -162,6 +183,17 @@ pub struct AuthContext { // None when the API runs open (no keys configured) pub key_name: Option, pub permissions: Permissions, + pub method: AuthMethod, +} + +// How the caller proved who they are: what the Users page shows as "you", +// and what decides which change revokes them. +#[derive(Clone, Copy, Debug, PartialEq)] +pub enum AuthMethod { + // No keys configured: everyone is admin. + Open, + ApiKey, + Ldap, } // The registry, injected as a router Extension. The middleware clones the @@ -236,6 +268,7 @@ pub async fn require_api_key( request.extensions_mut().insert(AuthContext { key_name: None, permissions: Permissions::Admin, + method: AuthMethod::Open, }); let mut response = next.run(request).await; // The console works here too. There is no ambient credential on an @@ -307,9 +340,16 @@ pub async fn require_api_key( key_name: auth.key_name.clone(), csrf: auth.csrf, }; + // On the request as well: `GET /auth` hands the token to a script + // (the UI), which has no HTML form to read it from. + request.extensions_mut().insert(identity.clone()); request.extensions_mut().insert(AuthContext { key_name: Some(auth.key_name), permissions: auth.permissions, + method: match auth.kind { + crate::adapters::r#in::http::session::SessionKind::ApiKey => AuthMethod::ApiKey, + crate::adapters::r#in::http::session::SessionKind::Directory => AuthMethod::Ldap, + }, }); let mut response = next.run(request).await; // On the RESPONSE, not the request: the HTML layer runs OUTSIDE this @@ -334,6 +374,7 @@ pub async fn require_api_key( request.extensions_mut().insert(AuthContext { key_name: Some(key.name.clone()), permissions: key.permissions.clone(), + method: AuthMethod::ApiKey, }); Ok(next.run(request).await) } @@ -344,6 +385,36 @@ pub async fn require_api_key( #[cfg(test)] mod tests { use super::*; + + // An LDAP-only instance would run open (no key = no authentication), so + // the combination is refused; one key is enough to close it. + #[test] + fn ldap_without_any_api_key_is_refused() { + let dir = tempfile::tempdir().unwrap(); + std::fs::write( + dir.path().join("config.yaml"), + "server:\n host: \"127.0.0.1\"\n port: 9090\n ui:\n enabled: true\n ldap:\n \ + url: \"ldaps://ldap.example.com\"\n bind_credential: cred-ldap\n search_base: \"DC=x\"\n \ + access:\n - name: ops\n groups: [DL-ops]\n role: admin\n", + ) + .unwrap(); + std::fs::write( + dir.path().join("credentials.yaml"), + "cred-ldap:\n name: \"LDAP\"\n type: username_password\n env_prefix: \"LDAP\"\n", + ) + .unwrap(); + let cfg = crate::config::load_config(dir.path().to_str().unwrap()).unwrap(); + + let err = ldap_requires_keys(&cfg, &[]).expect_err("no key with LDAP is refused"); + assert!(err.contains("no authentication"), "{err}"); + + let key = ResolvedApiKey { + name: "admin".to_string(), + secret: "s".to_string(), + permissions: Permissions::Admin, + }; + assert!(ldap_requires_keys(&cfg, &[key]).is_ok()); + } use axum::Router; use axum::body::Body; use axum::http::Request as HttpRequest; diff --git a/src/adapters/in/http/config.rs b/src/adapters/in/http/config.rs index d4b14cf..984df6a 100644 --- a/src/adapters/in/http/config.rs +++ b/src/adapters/in/http/config.rs @@ -346,6 +346,9 @@ fn keys_for( ) })?; + super::auth::ldap_requires_keys(cfg, &keys) + .map_err(|e| ApiError::new(StatusCode::CONFLICT, format!("refusing to reload: {e}")))?; + if keys.is_empty() && !current.0.is_empty() { return Err(ApiError::new( StatusCode::CONFLICT, diff --git a/src/adapters/in/http/credentials.rs b/src/adapters/in/http/credentials.rs new file mode 100644 index 0000000..f181a9c --- /dev/null +++ b/src/adapters/in/http/credentials.rs @@ -0,0 +1,120 @@ +use axum::Json; +use axum::extract::State; +use serde::Serialize; +use std::sync::Arc; +use utoipa::ToSchema; + +use crate::AppState; +use crate::adapters::r#in::http::auth::AuthContext; +use crate::adapters::r#in::http::error::{ApiError, ErrorBody}; +use crate::application::credentials::{check, users_of}; + +// The credentials configured on this instance, for the UI's Credentials +// page: what each one is, where its secrets come from, whether they resolve +// right now, and what depends on it. +// +// NEVER a secret value. The page answers "is it there, and who needs it" — +// the first questions when a sync fails with an authentication error, or +// before a credential is rotated or removed — and none of that needs a value. +// What it shows is what credentials.yaml already says (variable names, file +// paths, a Vault path) plus a yes/no. Admin-only all the same, like the +// configuration API that holds the same file: a map of where every secret +// lives is not for every key. + +#[derive(Serialize, ToSchema)] +pub struct CredentialInfo { + pub credential_id: String, + pub name: String, + /// `username_password`, `token` or `ssh_key` + #[serde(rename = "type")] + pub credential_type: &'static str, + /// Where the secrets are read from: `vault`, `env`, `file`, or `files` + /// (only file paths handed to the script) + pub backend: &'static str, + pub vault_path: Option, + /// The environment variables read, for an env-backed credential + pub env_vars: Vec, + pub secret_file: Option, + /// The names the connector receives the secrets under (not their values) + pub secret_keys: Vec, + /// Files handed to the script by path (an SSH key, a certificate) + pub file_keys: Vec, + /// Whether the secrets resolve right now, asked as a sync asks + pub resolves: bool, + /// Why not: the resolver's message, naming what is missing + pub error: Option, + pub used_by: CredentialUsersInfo, +} + +#[derive(Serialize, ToSchema)] +pub struct FileKeyInfo { + pub name: String, + pub path: String, +} + +#[derive(Serialize, ToSchema)] +pub struct CredentialUsersInfo { + pub sources: Vec, + pub projects: Vec, + /// Whether the LDAP login's service account binds with it + pub ldap: bool, +} + +#[utoipa::path( + get, + path = "/api/v1/credentials", + tag = "Credentials", + responses( + (status = 200, description = "Configured credentials: where their secrets come from, whether they resolve now, and what uses them. Never a secret value", body = Vec), + (status = 403, description = "API key is not admin", body = ErrorBody) + ) +)] +pub async fn list_credentials( + State(state): State>, + axum::Extension(auth): axum::Extension, +) -> Result>, ApiError> { + if !auth.permissions.is_admin() { + return Err(ApiError::admin_only()); + } + let config = state.config(); + + let mut credentials = Vec::with_capacity(config.credentials.len()); + for (id, credential) in &config.credentials { + let error = check(state.secrets.as_ref(), id).await; + let users = users_of(id, &config.sources, &config.projects, config.ldap.as_ref()); + + let mut secret_keys: Vec = credential.secret_keys.keys().cloned().collect(); + secret_keys.sort(); + let mut file_keys: Vec = credential + .file_keys + .iter() + .map(|(name, path)| FileKeyInfo { + name: name.clone(), + path: path.clone(), + }) + .collect(); + file_keys.sort_by(|a, b| a.name.cmp(&b.name)); + + credentials.push(CredentialInfo { + credential_id: id.clone(), + name: credential.name.clone(), + credential_type: credential.credential_type.name(), + backend: credential.backend(), + vault_path: credential.vault_path.clone(), + env_vars: credential.env_vars(), + secret_file: credential.secret_file.clone(), + secret_keys, + file_keys, + resolves: error.is_none(), + error, + used_by: CredentialUsersInfo { + sources: users.sources, + projects: users.projects, + ldap: users.ldap, + }, + }); + } + + credentials.sort_by(|a, b| a.credential_id.cmp(&b.credential_id)); + Ok(Json(credentials)) +} diff --git a/src/adapters/in/http/detail.rs b/src/adapters/in/http/detail.rs index 5cfbd25..6d8f456 100644 --- a/src/adapters/in/http/detail.rs +++ b/src/adapters/in/http/detail.rs @@ -81,6 +81,11 @@ pub struct Gathering { /// Credential ids this connector is given — names, never secrets pub credential_ids: Vec, pub allow_on_demand_refresh: bool, + /// For a remote source, the instance it pulls from (its `config.url`). + /// The one config value shown: it is an address, not a secret, and + /// `GET /topology` already names it to every caller who may read the source. + #[serde(skip_serializing_if = "Option::is_none")] + pub remote_url: Option, /// The KEYS of the connector's free-form `config:` map, without their /// values. A restricted key may read this page for a source it is granted, /// while the configuration API that holds the values is admin-only — so the @@ -146,6 +151,7 @@ pub async fn get_source( schedule: source.schedule.clone(), credential_ids: source.credential_ids.clone(), allow_on_demand_refresh: source.allow_on_demand_refresh, + remote_url: crate::domain::topology::remote_url(source), config_keys: { let mut keys: Vec = source.config.keys().cloned().collect(); keys.sort(); diff --git a/src/adapters/in/http/endpoints.rs b/src/adapters/in/http/endpoints.rs index bf0c605..87c7ec9 100644 --- a/src/adapters/in/http/endpoints.rs +++ b/src/adapters/in/http/endpoints.rs @@ -30,6 +30,30 @@ pub struct EndpointInfo { pub source_ids: Vec, pub sources_ready: usize, pub sources_missing: Vec, + /// How the sources' data becomes the response. Admin only, like a + /// source's `gathering`: a key granted an endpoint may read its output, + /// not how it is made. + #[serde(skip_serializing_if = "Option::is_none")] + pub transform: Option, +} + +/// An endpoint's transformer, as configured. Exactly one of `output` (a +/// builtin) and `script_path` is set. +#[derive(Serialize, ToSchema)] +pub struct EndpointTransform { + /// The builtin: `ansible`, `json` or `csv` + pub output: Option, + pub script_path: Option, + pub script_args: Vec, + /// Project whose checkout holds the script + pub project_id: Option, + /// Seconds a script may run before it is killed; null for a builtin, + /// which runs in-process and has no timeout + pub timeout_seconds: Option, + /// `limit.by_hosts_from_inventory`: only the hosts this source lists + pub limit_hosts_from: Option, + /// The KEYS of the transformer's `config:` map, without their values + pub config_keys: Vec, } #[utoipa::path( @@ -65,6 +89,27 @@ pub async fn list_endpoints( source_ids: ep.source_ids.clone(), sources_ready, sources_missing, + transform: auth.permissions.is_admin().then(|| EndpointTransform { + output: ep.output.map(|o| o.name().to_string()), + script_path: ep.script_path.clone(), + script_args: ep.script_args.clone(), + project_id: ep.project_id.clone(), + // The same default the run arms, so the page shows the + // timeout that actually applies. + timeout_seconds: ep.script_path.as_ref().map(|_| { + ep.timeout_seconds + .unwrap_or_else(crate::domain::default_timeout_seconds) + }), + limit_hosts_from: ep + .limit + .as_ref() + .and_then(|l| l.by_hosts_from_inventory.clone()), + config_keys: { + let mut keys: Vec = ep.config.keys().cloned().collect(); + keys.sort(); + keys + }, + }), } }) .collect(); @@ -73,6 +118,107 @@ pub async fn list_endpoints( Json(endpoints) } +/// How big an endpoint's default output is. +#[derive(Serialize, ToSchema)] +pub struct EndpointCounts { + /// Hosts in a call with no parameters of its own. Null for a script + /// endpoint: what it returns is the script's business, and counting its + /// inputs instead would pass off one number as another. + pub hosts: Option, + /// Groups in the same call; null for a script endpoint, likewise, and + /// for CSV, whose output has none. + pub groups: Option, + /// Sources with no data yet. While any is missing a call answers 503 + /// rather than an inventory, so there is nothing to count either: hosts + /// and groups are null, never a misleading zero. + pub missing_sources: Vec, +} + +#[utoipa::path( + get, + path = "/api/v1/endpoints/{id}/counts", + tag = "Endpoints", + params( + ("id" = String, Path, description = "Endpoint identifier (e.g. ep-ansible-linux)") + ), + responses( + (status = 200, description = "Hosts and groups in the endpoint's default output, from the sources cached now; the output itself is not rendered. Null counts with missing_sources when a call would answer 503", body = EndpointCounts), + (status = 400, description = "The builtin's configured settings are malformed, as a call would be told", body = ErrorBody), + (status = 403, description = "API key not allowed to run this endpoint", body = ErrorBody), + (status = 404, description = "Endpoint not configured", body = ErrorBody) + ) +)] +pub async fn endpoint_counts( + State(state): State>, + axum::Extension(auth): axum::Extension, + Path(id): Path, +) -> Result, ApiError> { + // Whoever may call the endpoint may know how big its answer is. + if !auth.permissions.allows_endpoint(&id) { + return Err(ApiError::forbidden(format!( + "this API key is not allowed to run endpoint '{}'", + id + ))); + } + let config = state.config(); + let endpoint = config + .endpoints + .get(&id) + .ok_or_else(|| ApiError::not_found(format!("endpoint '{}' is not configured", id)))?; + + // What a call would merge right now: the sources cached at this moment. + let mut datasets: HashMap> = HashMap::new(); + let mut missing_sources: Vec = Vec::new(); + for source_id in &endpoint.source_ids { + match state.cache.get(source_id) { + Some(entry) => { + datasets.insert(source_id.clone(), entry.dataset.clone()); + } + None => missing_sources.push(source_id.clone()), + } + } + missing_sources.sort(); + + // Settings the builtin would refuse (a malformed filter, a renamed key) + // fail a call with 400; the counts answer the same rather than a zero. + if endpoint.output.is_some() { + crate::application::output::check_params(&endpoint.config, &serde_json::json!({})) + .map_err(ApiError::bad_request)?; + } + + // A script's output, or a call that would fail for want of a source: + // nothing to count. + if endpoint.output.is_none() || !missing_sources.is_empty() { + return Ok(Json(EndpointCounts { + hosts: None, + groups: None, + missing_sources, + })); + } + + let datasets = match &endpoint.limit { + Some(limit) => crate::application::output::apply_limit(&datasets, limit), + None => datasets, + }; + + // The merge is CPU over the whole inventory: off the async workers, as + // a render is. + let settings = endpoint.config.clone(); + let (hosts, groups) = tokio::task::spawn_blocking(move || { + crate::application::output::counts(&datasets, &settings) + }) + .await + .map_err(|join_error| ApiError::internal(format!("counting failed: {}", join_error)))?; + + // CSV is one row per host: its output has no groups to count. + let csv = endpoint.output == Some(crate::domain::endpoint::OutputFormat::Csv); + Ok(Json(EndpointCounts { + hosts: Some(hosts), + groups: (!csv).then_some(groups), + missing_sources, + })) +} + #[utoipa::path( post, path = "/api/v1/endpoints/{id}", diff --git a/src/adapters/in/http/host_search.rs b/src/adapters/in/http/host_search.rs new file mode 100644 index 0000000..006e4bd --- /dev/null +++ b/src/adapters/in/http/host_search.rs @@ -0,0 +1,103 @@ +use axum::Json; +use axum::extract::{Query, State}; +use serde::{Deserialize, Serialize}; +use std::sync::Arc; +use utoipa::{IntoParams, ToSchema}; + +use crate::AppState; +use crate::adapters::r#in::http::auth::AuthContext; +use crate::adapters::r#in::http::error::{ApiError, ErrorBody}; +use crate::application::host_search::search; + +// Declared as query parameters here: the handler takes them wrapped in a +// Result (to answer a malformed one as JSON), which hides the Query the +// OpenAPI derive would otherwise infer them from. +#[derive(Deserialize, IntoParams)] +#[into_params(parameter_in = Query)] +pub struct HostSearchQuery { + /// Part of a host name, any case; at least two characters + pub q: String, + /// Most hosts to return (default 20, at most 100) + pub limit: Option, +} + +#[derive(Serialize, ToSchema)] +pub struct HostSearchResult { + /// The hosts found, best match first: the whole name, then its start, + /// then anywhere in it + pub hosts: Vec, + /// How many hosts matched in all, before the limit + pub total: usize, +} + +#[derive(Serialize, ToSchema)] +pub struct FoundHost { + pub host: String, + /// The sources holding it, with the age of each one's data + pub sources: Vec, + /// The endpoints whose default output includes it: their sources, limit + /// and own settings decide, and an endpoint missing data from a source + /// (which answers 503) is never listed. A caller's own parameters can + /// still leave the host out of a particular call. + pub endpoints: Vec, +} + +#[derive(Serialize, ToSchema)] +pub struct FoundIn { + pub source_id: String, + pub age_seconds: u64, +} + +#[utoipa::path( + get, + path = "/api/v1/hosts/search", + tag = "Sources", + params(HostSearchQuery), + responses( + (status = 200, description = "Hosts whose name contains the query, across every source the key may read, with where each is held and which endpoints it reaches. From the cache: nothing is gathered", body = HostSearchResult), + (status = 400, description = "`q` is missing or a parameter is malformed", body = ErrorBody), + (status = 401, description = "Not authenticated", body = ErrorBody) + ) +)] +pub async fn search_hosts( + State(state): State>, + axum::Extension(auth): axum::Extension, + query: Result, axum::extract::rejection::QueryRejection>, +) -> Result, ApiError> { + // A missing `q` answers like every other failure: {"error": ...}. + let Query(query) = query.map_err(|e| ApiError::bad_request(e.body_text()))?; + let config = state.config(); + let limit = query.limit.unwrap_or(20).clamp(1, 100); + let result = search( + &query.q, + limit, + crate::application::host_search::Catalog { + sources: &config.sources, + views: &config.views, + endpoints: &config.endpoints, + cache: state.cache.as_ref(), + scopes: state.advertised_scopes.as_ref(), + }, + |id| auth.permissions.allows_source(id), + |id| auth.permissions.allows_endpoint(id), + ); + Ok(Json(HostSearchResult { + hosts: result + .matches + .into_iter() + .map(|m| FoundHost { + host: m.host, + sources: m + .sources + .into_iter() + .map(|s| FoundIn { + source_id: s.source_id, + age_seconds: s.age_seconds, + }) + .collect(), + endpoints: m.endpoints, + }) + .collect(), + total: result.total, + })) +} diff --git a/src/adapters/in/http/html.rs b/src/adapters/in/http/html.rs index fa31cc6..ef74159 100644 --- a/src/adapters/in/http/html.rs +++ b/src/adapters/in/http/html.rs @@ -106,12 +106,14 @@ pub const THEMES: [&str; 6] = ["auto", "light", "dark", "dracula", "nord", "sola const THEME_COOKIE: &str = "uapi_theme"; pub fn palette_css(theme: &str) -> String { - // bg, fg, dim, key, str, num, link, rule, panel - let light = "--bg:#fbfbfa;--fg:#24292f;--dim:#6e7781;--key:#0550ae;--str:#0a6847;--num:#953800;--link:#0969da;--rule:#d8dee4;--panel:#fff;"; - let dark = "--bg:#0d1117;--fg:#e6edf3;--dim:#8b949e;--key:#79c0ff;--str:#7ee787;--num:#ffa657;--link:#58a6ff;--rule:#30363d;--panel:#161b22;"; - let dracula = "--bg:#282a36;--fg:#f8f8f2;--dim:#6272a4;--key:#8be9fd;--str:#50fa7b;--num:#bd93f9;--link:#ff79c6;--rule:#44475a;--panel:#21222c;"; - let nord = "--bg:#2e3440;--fg:#eceff4;--dim:#7b88a1;--key:#88c0d0;--str:#a3be8c;--num:#d08770;--link:#81a1c1;--rule:#434c5e;--panel:#3b4252;"; - let solarized = "--bg:#002b36;--fg:#93a1a1;--dim:#586e75;--key:#268bd2;--str:#859900;--num:#cb4b16;--link:#2aa198;--rule:#073642;--panel:#073642;"; + // bg, fg, dim, key, str, num, link, rule, panel, backdrop (behind the + // login card: a step darker than bg, so the card reads as the one thing + // on the page) + let light = "--bg:#fbfbfa;--fg:#24292f;--dim:#6e7781;--key:#0550ae;--str:#0a6847;--num:#953800;--link:#0969da;--rule:#d8dee4;--panel:#fff;--backdrop:#e6eaef;"; + let dark = "--bg:#0d1117;--fg:#e6edf3;--dim:#8b949e;--key:#79c0ff;--str:#7ee787;--num:#ffa657;--link:#58a6ff;--rule:#30363d;--panel:#161b22;--backdrop:#010409;"; + let dracula = "--bg:#282a36;--fg:#f8f8f2;--dim:#6272a4;--key:#8be9fd;--str:#50fa7b;--num:#bd93f9;--link:#ff79c6;--rule:#44475a;--panel:#21222c;--backdrop:#191a21;"; + let nord = "--bg:#2e3440;--fg:#eceff4;--dim:#7b88a1;--key:#88c0d0;--str:#a3be8c;--num:#d08770;--link:#81a1c1;--rule:#434c5e;--panel:#3b4252;--backdrop:#242933;"; + let solarized = "--bg:#002b36;--fg:#93a1a1;--dim:#586e75;--key:#268bd2;--str:#859900;--num:#cb4b16;--link:#2aa198;--rule:#073642;--panel:#073642;--backdrop:#001e26;"; match theme { "light" => format!(":root{{{light}}}"), diff --git a/src/adapters/in/http/index.rs b/src/adapters/in/http/index.rs index 2d1fba7..e7627cd 100644 --- a/src/adapters/in/http/index.rs +++ b/src/adapters/in/http/index.rs @@ -81,6 +81,14 @@ pub struct ApiIndex { /// Admin-only, and only when config_api.enabled is on #[serde(skip_serializing_if = "Option::is_none")] pub config: Option<&'static str>, + /// Who this caller is, and how anyone can log in + pub auth: &'static str, + /// The instances this one pulls from, and theirs + pub topology: &'static str, + /// Configured credentials and whether they resolve (admin-only) + pub credentials: &'static str, + /// The configuration in force, defaults included (admin-only) + pub settings: &'static str, pub health: &'static str, pub ready: &'static str, pub metrics: &'static str, @@ -114,6 +122,10 @@ pub async fn api_index( // Same condition the config handlers enforce: the routes exist in the // router either way, and answer 403 when no store is configured. config: (admin && state.config_store.is_some()).then_some("/api/v1/config"), + auth: "/api/v1/auth", + topology: "/api/v1/topology", + credentials: "/api/v1/credentials", + settings: "/api/v1/settings", health: "/healthz", ready: "/readyz", metrics: "/metrics", diff --git a/src/adapters/in/http/login.rs b/src/adapters/in/http/login.rs index 1a3a003..a34072c 100644 --- a/src/adapters/in/http/login.rs +++ b/src/adapters/in/http/login.rs @@ -1,22 +1,27 @@ -use axum::extract::Form; +use axum::extract::{Form, Query, State}; use axum::http::{HeaderValue, StatusCode, header}; use axum::response::{IntoResponse, Response}; use serde::Deserialize; +use std::sync::Arc; use std::time::Duration; -use crate::adapters::r#in::http::auth::{ApiKeys, match_token}; +use crate::AppState; +use crate::adapters::r#in::http::auth::{ApiKeys, Permissions, match_token}; use crate::adapters::r#in::http::html::{Html, THEMES}; -use crate::adapters::r#in::http::session::{COOKIE_NAME, Sessions, cookie_value}; +use crate::adapters::r#in::http::session::{COOKIE_NAME, SessionKind, Sessions, cookie_value}; +use crate::application::login::{LoginError, throttled_ldap_login}; +use crate::domain::ldap::Grant; // The login form for the browsable API. // // Public routes, outside the key middleware — a page that asks for a credential // cannot itself require one. They exist only when the UI is enabled. // -// What is validated is an API KEY, against the same registry and the same -// constant-time compare a header goes through. There is no second credential -// store, no user table and no password: the console is a way to use a key you -// already have from a browser, which cannot send headers. +// Two ways in, and neither is a credential store of its own. An API KEY is +// checked against the same registry and the same constant-time compare a +// header goes through. A directory account (when `server.ui.ldap` is set) is +// checked by the directory, and what it may touch comes from the configured +// access rules. There is no user table and no password stored here. // A short-lived token planted when the form is served and required back when // it is submitted, so only a form THIS server rendered can log anybody in. @@ -28,13 +33,54 @@ use crate::adapters::r#in::http::session::{COOKIE_NAME, Sessions, cookie_value}; // and is audit-logged — under someone else's key without noticing. const LOGIN_COOKIE: &str = "uapi_login"; +// One form, two shapes: `key` for an API key, `username` + `password` for +// LDAP. Whichever is filled in decides the path. #[derive(Deserialize)] pub struct LoginForm { - pub key: String, + #[serde(default)] + pub key: Option, + #[serde(default)] + pub username: Option, + #[serde(default)] + pub password: Option, + #[serde(default)] + pub next: Option, #[serde(default)] pub _csrf: String, } +#[derive(Deserialize)] +pub struct LoginQuery { + #[serde(default)] + pub next: Option, +} + +// Where to go after logging in: the page that sent the browser here, or +// `server.ui.home` when none did. Only a path on THIS origin: +// `next=https://evil.example` (or `//evil.example`, which a browser reads as +// another host) would turn the login into an open redirect that lends our +// address to a phishing link. +fn safe_next<'a>(next: Option<&'a str>, home: &'a str) -> &'a str { + match next { + Some(path) if crate::config::is_local_path(path) => path, + _ => home, + } +} + +// For a `next` carried in a query string: every byte but the unreserved ones +// and '/' escaped, so a path with its own query ("?host=a&b") arrives whole. +fn query_escape(value: &str) -> String { + value + .bytes() + .map(|b| match b { + b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'.' | b'_' | b'~' | b'/' => { + (b as char).to_string() + } + _ => format!("%{b:02X}"), + }) + .collect() +} + fn esc(s: &str) -> String { Html::text(s).as_str().to_string() } @@ -42,16 +88,87 @@ fn esc(s: &str) -> String { // Rendered by hand rather than through the JSON page: there is no resource here // and nothing to negotiate. It borrows the same palette so the login does not // look like a different product. -fn login_page( - theme: &str, - message: Option<&str>, +struct Page<'a> { + theme: &'a str, + message: Option<&'a str>, status: StatusCode, - challenge: Option<&str>, -) -> Response { - let notice = match message { + challenge: Option<&'a str>, + ldap: bool, + // Which way in the card opens on: the one the person just tried, so a + // mistyped key does not drop them back on the directory form. + key_first: bool, + next: &'a str, +} + +fn login_page(page: Page<'_>) -> Response { + let notice = match page.message { Some(text) => format!("

{}

", esc(text)), None => String::new(), }; + let challenge = esc(page.challenge.unwrap_or("")); + let next = esc(page.next); + + let hidden = format!( + r#" + "# + ); + let key_form = |autofocus: bool| { + format!( + r#"
+ {hidden} + + + +
"#, + autofocus = if autofocus { " autofocus" } else { "" }, + ) + }; + + // With LDAP, one card and a switch between the two ways in. The switch is + // a pair of radio buttons and `:has()` shows the form that matches: no + // script, so the page keeps its strict CSP. The radios sit outside both + // forms, so neither submits them. + let content = if page.ldap { + let (ldap_checked, key_checked) = if page.key_first { + ("", " checked") + } else { + (" checked", "") + }; + format!( + r#"
+ + + + +
+
+ {hidden} + + + + + +
+ {key}"#, + ldap_focus = if page.key_first { "" } else { " autofocus" }, + key = key_form(page.key_first), + ) + } else { + key_form(true) + }; + let lead = if page.ldap { + "Sign in" + } else { + "Sign in with an API key" + }; + let forms = format!( + r#"
+

unified-api

+

{lead}

+ {notice} + {content} +
"# + ); let body = format!( r#" @@ -59,58 +176,89 @@ fn login_page( -log in — unified-api +Sign in — unified-api -
-

unified-api

-

Sign in with an API key to browse from this browser.

- {notice} - - - - -

A key sent as X-API-Key needs no session. - OpenAPI

-
+{forms} "#, - palette = crate::adapters::r#in::http::html::palette_css(theme), - notice = notice, - challenge = esc(challenge.unwrap_or("")), + palette = crate::adapters::r#in::http::html::palette_css(page.theme), ); ( - status, + page.status, [ ( header::CONTENT_TYPE, @@ -143,9 +291,27 @@ fn theme_of(headers: &axum::http::HeaderMap) -> String { .to_string() } -pub async fn login_form(headers: axum::http::HeaderMap) -> Response { +pub async fn login_form( + State(state): State>, + Query(query): Query, + headers: axum::http::HeaderMap, +) -> Response { + let config = state.config(); let challenge = crate::adapters::r#in::http::session::new_token(); - let mut response = login_page(&theme_of(&headers), None, StatusCode::OK, Some(&challenge)); + let response = login_page(Page { + theme: &theme_of(&headers), + message: None, + status: StatusCode::OK, + challenge: Some(&challenge), + ldap: config.ldap.is_some(), + key_first: false, + next: safe_next(query.next.as_deref(), &config.ui_home), + }); + with_challenge(response, &challenge) +} + +// Plants the cookie half of the login challenge; the form carries the other. +fn with_challenge(mut response: Response, challenge: &str) -> Response { if let Ok(value) = HeaderValue::from_str(&format!( "{LOGIN_COOKIE}={challenge}; Path=/api/v1/login; HttpOnly; SameSite=Strict; Max-Age=600" )) { @@ -155,6 +321,7 @@ pub async fn login_form(headers: axum::http::HeaderMap) -> Response { } pub async fn login_submit( + State(state): State>, axum::Extension(keys): axum::Extension, axum::Extension(sessions): axum::Extension, axum::Extension(ttl): axum::Extension, @@ -162,6 +329,28 @@ pub async fn login_submit( Form(form): Form, ) -> Response { let theme = theme_of(&headers); + let config = state.config(); + let ldap = config.ldap.clone(); + let next = safe_next(form.next.as_deref(), &config.ui_home).to_string(); + // A refused login re-renders the form with a FRESH challenge. Without + // one, the page that says "wrong password" carried no token, and the + // corrected attempt was rejected as an expired form. + let key_first = form.username.is_none(); + let refuse = |message: &str, status: StatusCode| { + let challenge = crate::adapters::r#in::http::session::new_token(); + with_challenge( + login_page(Page { + theme: &theme, + message: Some(message), + status, + challenge: Some(&challenge), + ldap: ldap.is_some(), + key_first, + next: &next, + }), + &challenge, + ) + }; // The form must be one this server handed out. Both halves of the pair // have to agree: the cookie planted when the form was rendered, and the @@ -177,48 +366,102 @@ pub async fn login_submit( }); if !matched { - return login_page( - &theme, - Some("this login form has expired — reload the page and try again"), + return refuse( + "this login form has expired — reload the page and try again", StatusCode::FORBIDDEN, - None, ); } - let registry = keys.0.load(); - - // An open API has nothing to log into: every caller is already admin, and a - // form that accepts anything would suggest otherwise. - if registry.is_empty() { - // Deliberately the same answer a wrong key gets. Saying "this instance - // has no authentication" to an unauthenticated caller is a banner for - // anyone scanning; the real reason goes to the log, where an operator - // can see it and a scanner cannot. - tracing::warn!("login attempted on an instance with no API keys configured"); - return login_page( - &theme, - Some("invalid API key"), - StatusCode::UNAUTHORIZED, - None, - ); - } + let (name, kind, permissions) = if let Some(username) = form.username.as_deref() { + let Some(ldap) = ldap.as_ref() else { + return refuse("LDAP login is not configured", StatusCode::BAD_REQUEST); + }; + let password = form.password.as_deref().unwrap_or(""); + match throttled_ldap_login( + state.login_throttle.as_ref(), + state.secrets.as_ref(), + state.directory.as_ref(), + ldap, + username, + password, + ) + .await + { + Ok(logged) => ( + format!("ldap:{}", logged.username), + SessionKind::Directory, + match logged.grant { + Grant::Admin => Permissions::Admin, + Grant::Scoped { sources, endpoints } => { + Permissions::Scoped { sources, endpoints } + } + }, + ), + Err(LoginError::InvalidCredentials) => { + // Already slowed to a fixed total time by the throttle, so an + // unknown user and a wrong password look alike. + return refuse("invalid username or password", StatusCode::UNAUTHORIZED); + } + Err(LoginError::TooManyAttempts) => { + return refuse( + "too many failed attempts for this account — try again in 15 minutes, \ + or sign in with an API key", + StatusCode::TOO_MANY_REQUESTS, + ); + } + Err(LoginError::Busy) => { + return refuse( + "too many sign-ins at once — try again in a moment", + StatusCode::SERVICE_UNAVAILABLE, + ); + } + Err(LoginError::NotAuthorized) => { + // The password was right, so saying so leaks nothing a + // guesser could use, and saves the person a ticket. + return refuse( + "your account is valid but has no access to this instance — ask an \ + administrator to add you or your group to server.ui.ldap.access", + StatusCode::FORBIDDEN, + ); + } + Err(LoginError::Unavailable(reason)) => { + tracing::error!(%reason, "LDAP login failed: directory unavailable"); + return refuse( + "the directory could not be reached — try again shortly, or sign in \ + with an API key", + StatusCode::SERVICE_UNAVAILABLE, + ); + } + } + } else { + let registry = keys.0.load(); - let Some(key) = match_token(form.key.trim(), ®istry) else { - // No detail, deliberately: "no such key" and "wrong key" are the same - // sentence to someone guessing. - return login_page( - &theme, - Some("invalid API key"), - StatusCode::UNAUTHORIZED, - None, - ); + // An open API has nothing to log into: every caller is already admin, and a + // form that accepts anything would suggest otherwise. + if registry.is_empty() { + // Deliberately the same answer a wrong key gets. Saying "this instance + // has no authentication" to an unauthenticated caller is a banner for + // anyone scanning; the real reason goes to the log, where an operator + // can see it and a scanner cannot. + tracing::warn!("login attempted on an instance with no API keys configured"); + return refuse("invalid API key", StatusCode::UNAUTHORIZED); + } + + let Some(key) = match_token(form.key.as_deref().unwrap_or("").trim(), ®istry) else { + // No detail, deliberately: "no such key" and "wrong key" are the same + // sentence to someone guessing. + return refuse("invalid API key", StatusCode::UNAUTHORIZED); + }; + ( + key.name.clone(), + SessionKind::ApiKey, + key.permissions.clone(), + ) }; - let issued = sessions - .0 - .create(key.name.clone(), key.permissions.clone(), ttl.0); + let issued = sessions.0.create(name.clone(), kind, permissions, ttl.0); - tracing::info!(key_name = %key.name, "browser session opened"); + tracing::info!(key_name = %name, "browser session opened"); // Secure is NOT set here: the console is reachable over plain HTTP in a // port-forward or a local run, and a Secure cookie would silently never be @@ -233,16 +476,21 @@ pub async fn login_submit( ttl.0.as_secs() ); - let mut response = axum::response::Redirect::to("/api/v1/").into_response(); + let mut response = axum::response::Redirect::to(&next).into_response(); if let Ok(value) = HeaderValue::from_str(&cookie) { response.headers_mut().insert(header::SET_COOKIE, value); } response } +// A sign-out form may say where the next sign-in should land (the UI posts +// next=/), so signing straight back in returns to the UI rather than to +// `server.ui.home`. Optional: a bare POST, with no form at all, still signs +// out. pub async fn logout( axum::Extension(sessions): axum::Extension, headers: axum::http::HeaderMap, + form: Result, axum::extract::rejection::FormRejection>, ) -> Response { if let Some(token) = headers .get(header::COOKIE) @@ -254,7 +502,13 @@ pub async fn logout( sessions.0.remove(token); } - let mut response = axum::response::Redirect::to("/api/v1/login").into_response(); + let login = match form.ok().and_then(|Form(f)| f.next) { + Some(next) if crate::config::is_local_path(&next) => { + format!("/api/v1/login?next={}", query_escape(&next)) + } + _ => "/api/v1/login".to_string(), + }; + let mut response = axum::response::Redirect::to(&login).into_response(); response.headers_mut().insert( header::SET_COOKIE, HeaderValue::from_static("uapi_session=; Path=/; HttpOnly; SameSite=Strict; Max-Age=0"), diff --git a/src/adapters/in/http/metrics.rs b/src/adapters/in/http/metrics.rs index bdaff5f..461a6dd 100644 --- a/src/adapters/in/http/metrics.rs +++ b/src/adapters/in/http/metrics.rs @@ -1,3 +1,4 @@ +use std::borrow::Cow; use std::collections::HashSet; use std::sync::{Arc, OnceLock}; @@ -8,6 +9,7 @@ use metrics_exporter_prometheus::{PrometheusBuilder, PrometheusHandle}; use crate::AppState; use crate::adapters::r#in::http::auth::{ApiKeys, match_token, presented_token}; use crate::adapters::r#in::http::error::ApiError; +use crate::domain::dataset::Dataset; // The metrics recorder is a process-wide global (like the tracing subscriber), // so it can only be installed once. OnceLock makes repeated AppBuilder::build() @@ -92,29 +94,189 @@ pub async fn track_requests( scrape-time gauges (see docs/observability.md). Public by default; \ `server.metrics_require_auth: true` requires the API key, since the \ exposition labels every source id and host count", content_type = "text/plain"), - (status = 401, description = "server.metrics_require_auth is on and no valid API key \ - was presented", body = crate::adapters::r#in::http::error::ErrorBody) + (status = 401, description = "server.metrics_require_auth is on and neither a valid API \ + key nor a signed-in session was presented", body = crate::adapters::r#in::http::error::ErrorBody) ) )] pub async fn metrics( State(state): State>, axum::Extension(registry): axum::Extension, + sessions: Option>, headers: HeaderMap, ) -> Result { if state.config().metrics_require_auth { let keys = registry.0.load(); if !keys.is_empty() { - let token = presented_token(&headers).ok_or_else(ApiError::missing_api_key)?; - match_token(token, &keys).ok_or_else(ApiError::invalid_api_key)?; + match presented_token(&headers) { + Some(token) => { + match_token(token, &keys).ok_or_else(ApiError::invalid_api_key)?; + } + // No key in a header: a browser signed in to this instance + // reads it with its session, like any other route. A read, + // so no CSRF token is asked for. + None => { + use crate::adapters::r#in::http::session::{COOKIE_NAME, cookie_value}; + let signed_in = sessions.is_some_and(|axum::Extension(sessions)| { + headers + .get(axum::http::header::COOKIE) + .and_then(|v| v.to_str().ok()) + .and_then(|cookies| cookie_value(cookies, COOKIE_NAME)) + .and_then(|token| sessions.0.get(token)) + .is_some() + }); + if !signed_in { + return Err(ApiError::missing_api_key()); + } + } + } } } record_source_gauges(&state); record_task_health_gauges(&state); record_config_gauges(&state); + record_process_gauges(); Ok(handle().render()) } +// Every hostname of every source, counted once, ignoring case: DNS does +// not distinguish `AASQL101` from `aasql101`, and sources disagree on it. +// Only a name that actually has capitals is copied to lowercase it; the rest +// stay borrowed. +// +// Case is the only folding done here. Merging `web01` into +// `web01.example.com`, or one domain into another, would need a rule that +// holds for every estate, and none does: `accesscontrol1` exists as three +// different machines in three datacenters. +fn unique_hosts(datasets: &[&Dataset]) -> usize { + datasets + .iter() + .flat_map(|dataset| dataset.hostvars.keys()) + .map(|name| { + if name.bytes().any(|b| b.is_ascii_uppercase()) { + Cow::Owned(name.to_ascii_lowercase()) + } else { + Cow::Borrowed(name.as_str()) + } + }) + .collect::>>() + .len() +} + +// The process's own resource use, under the standard Prometheus names +// (process_resident_memory_bytes, process_cpu_seconds_total, ...) so stock +// dashboards and alerts work unchanged. +// +// Memory matters more here than in most services: the whole inventory cache +// lives in it, so resident memory is the number that says when a pod is +// about to meet its limit. +// +// Read from /proc at scrape time rather than through a collector crate: the +// files are a few hundred bytes, the reads are cheap enough for a blocking +// call on a scrape, and it adds nothing to the dependency tree. Linux only: +// that is where this runs (containers), and elsewhere the series are simply +// absent rather than wrong. +#[cfg(target_os = "linux")] +fn record_process_gauges() { + if let Ok(status) = std::fs::read_to_string("/proc/self/status") { + let stats = parse_proc_status(&status); + if let Some(rss) = stats.resident_bytes { + metrics::gauge!("process_resident_memory_bytes").set(rss as f64); + } + if let Some(virt) = stats.virtual_bytes { + metrics::gauge!("process_virtual_memory_bytes").set(virt as f64); + } + if let Some(threads) = stats.threads { + metrics::gauge!("process_threads").set(threads as f64); + } + } + + // A gauge under a counter's name: the exporter's counters are integers, + // and CPU time in whole seconds would make rate() useless at the scale of + // a quiet service. rate() works the same on either type. + if let Ok(stat) = std::fs::read_to_string("/proc/self/stat") + && let Some(seconds) = parse_stat_cpu_seconds(&stat) + { + metrics::gauge!("process_cpu_seconds_total").set(seconds); + } + + // Less the one read_dir itself holds open while it lists the directory. + if let Ok(entries) = std::fs::read_dir("/proc/self/fd") { + metrics::gauge!("process_open_fds").set(entries.count().saturating_sub(1) as f64); + } + if let Ok(limits) = std::fs::read_to_string("/proc/self/limits") + && let Some(max) = parse_max_open_files(&limits) + { + metrics::gauge!("process_max_fds").set(max as f64); + } +} + +#[cfg(not(target_os = "linux"))] +fn record_process_gauges() {} + +#[derive(Debug, Default, PartialEq)] +struct ProcStatus { + resident_bytes: Option, + virtual_bytes: Option, + threads: Option, +} + +// `/proc/self/status` reports memory in kB ("VmRSS: 123456 kB"), which +// sidesteps the page size that /proc/self/statm would need. +fn parse_proc_status(text: &str) -> ProcStatus { + let mut stats = ProcStatus::default(); + for line in text.lines() { + let Some((key, rest)) = line.split_once(':') else { + continue; + }; + let number = rest + .split_whitespace() + .next() + .and_then(|n| n.parse::().ok()); + match key { + "VmRSS" => stats.resident_bytes = number.map(|kb| kb * 1024), + "VmSize" => stats.virtual_bytes = number.map(|kb| kb * 1024), + "Threads" => stats.threads = number, + _ => {} + } + } + stats +} + +// User plus system CPU time of the whole process, every thread, from +// `/proc/self/stat` (fields 14 and 15). Not /proc/self/schedstat: that file +// is per TASK, so it counts only the main thread, while the work runs on the +// runtime's worker threads, and the rate read close to zero under load. +// +// The values are in clock ticks of USER_HZ, which Linux fixes at 100 for +// everything it reports under /proc regardless of the kernel's own tick rate. +// The process name (field 2) is in parentheses and may itself contain spaces +// or parentheses, so the fields are counted from after the LAST ')'. +const USER_HZ: f64 = 100.0; + +fn parse_stat_cpu_seconds(text: &str) -> Option { + let after_name = &text[text.rfind(')')? + 1..]; + // Field 3 (state) is the first after the name: utime is field 14 and + // stime field 15, so 11 and 12 here. + let mut fields = after_name.split_whitespace().skip(11); + let utime: u64 = fields.next()?.parse().ok()?; + let stime: u64 = fields.next()?.parse().ok()?; + Some((utime + stime) as f64 / USER_HZ) +} + +// The soft limit from the "Max open files" row of /proc/self/limits. +// "unlimited" has no number to report, so the series is left out. +fn parse_max_open_files(text: &str) -> Option { + let row = text + .lines() + .find(|line| line.starts_with("Max open files"))?; + row.trim_start_matches("Max open files") + .split_whitespace() + .next()? + .parse() + .ok() +} + // Which build and which configuration this process is running — the first // questions a fleet dashboard asks now that a pod's configuration can change // without a restart. Scrape-time like everything here. @@ -184,6 +346,9 @@ fn record_source_gauges(state: &AppState) { // Driven by the cache, not by config: an entry can outlive its config // entry (source removed from YAML, cache still serving it) and that is // exactly the state an operator wants to see. + // Kept until the loop ends, so the unique count below can borrow every + // hostname instead of copying ~tens of thousands of strings per scrape. + let mut entries = Vec::with_capacity(cached.len()); for source_id in cached { let Some(entry) = state.cache.get(&source_id) else { // Evicted between keys() and get() — it will show up next scrape @@ -200,8 +365,19 @@ fn record_source_gauges(state: &AppState) { .set(entry.dataset.hostvars.len() as f64); metrics::gauge!("unified_api_source_groups", "source" => source_id.clone()) .set(entry.dataset.groups.len() as f64); + entries.push(entry); } + // The number of unique machines the inventory knows. Summing + // unified_api_source_hosts counts a host once per source it appears in + // (Device42, vCenter, monitoring and SSH facts all list the same box), so + // the sum overstates the estate several times over. This is the union of + // hostnames across every cached source: exact names, so a host listed as + // `web01` in one source and `web01.example.com` in another still counts + // twice, which is a naming difference worth seeing rather than hiding. + let datasets: Vec<&Dataset> = entries.iter().map(|entry| entry.dataset.as_ref()).collect(); + metrics::gauge!("unified_api_hosts_unique").set(unique_hosts(&datasets) as f64); + record_view_gauges(state); } @@ -323,3 +499,65 @@ fn record_task_health_gauges(state: &AppState) { pub fn init() { handle(); } + +#[cfg(test)] +mod process_tests { + use super::*; + + // The estate, not the sum: a host listed by three sources is one host, + // whatever case each source writes it in. + #[test] + fn a_host_in_several_sources_counts_once() { + let parse = |json: &str| serde_json::from_str::(json).unwrap(); + let d42 = parse(r#"{"hostvars": {"web01": {}, "db01": {}}}"#); + let vmware = parse(r#"{"hostvars": {"web01": {}, "app01": {}}}"#); + let facts = parse(r#"{"hostvars": {"WEB01": {}, "web01.example.com": {}}}"#); + // web01 (in any case), db01, app01, and the FQDN spelling as its own + // name: only case is folded, never domains + assert_eq!(unique_hosts(&[&d42, &vmware, &facts]), 4); + assert_eq!(unique_hosts(&[]), 0); + } + + #[test] + fn status_memory_is_reported_in_bytes() { + let status = "Name:\tunified-api\nVmSize:\t 2048 kB\nVmRSS:\t 512 kB\nThreads:\t17\n"; + assert_eq!( + parse_proc_status(status), + ProcStatus { + resident_bytes: Some(512 * 1024), + virtual_bytes: Some(2048 * 1024), + threads: Some(17), + } + ); + } + + #[test] + fn stat_cpu_time_is_user_plus_system_in_seconds() { + // A name with spaces and a ')' of its own must not shift the fields. + let line = "4242 (tokio rt) worker)) S 1 4242 4242 0 -1 4194560 100 0 0 0 \ + 250 150 0 0 20 0 9 0 12345 0 0"; + assert_eq!(parse_stat_cpu_seconds(line), Some(4.0)); + assert_eq!(parse_stat_cpu_seconds(""), None); + assert_eq!(parse_stat_cpu_seconds("1 (x) S"), None); + } + + #[test] + fn max_open_files_takes_the_soft_limit() { + let limits = "Limit Soft Limit Hard Limit Units\n\ + Max open files 1024 524288 files\n"; + assert_eq!(parse_max_open_files(limits), Some(1024)); + let unlimited = + "Max open files unlimited unlimited files\n"; + assert_eq!(parse_max_open_files(unlimited), None); + } + + // On the platform this ships on, the real files parse into real numbers. + #[cfg(target_os = "linux")] + #[test] + fn this_process_reports_its_own_memory() { + let status = std::fs::read_to_string("/proc/self/status").unwrap(); + assert!(parse_proc_status(&status).resident_bytes.unwrap() > 0); + let stat = std::fs::read_to_string("/proc/self/stat").unwrap(); + assert!(parse_stat_cpu_seconds(&stat).is_some()); + } +} diff --git a/src/adapters/in/http/mod.rs b/src/adapters/in/http/mod.rs index c47698c..a8ecbba 100644 --- a/src/adapters/in/http/mod.rs +++ b/src/adapters/in/http/mod.rs @@ -4,11 +4,13 @@ pub mod audit; pub mod auth; pub mod cache; pub mod config; +pub mod credentials; pub mod detail; pub mod endpoints; pub mod enrichers; pub mod error; pub mod health; +pub mod host_search; pub mod hosts; pub mod html; pub mod index; @@ -19,7 +21,11 @@ pub mod projects; pub mod routes; pub mod scope; pub mod session; +pub mod settings; pub mod sources; pub mod spec; pub mod sync; +pub mod topology; +pub mod ui_assets; +pub mod users; pub mod views; diff --git a/src/adapters/in/http/openapi.rs b/src/adapters/in/http/openapi.rs index 9a17942..d5816fa 100644 --- a/src/adapters/in/http/openapi.rs +++ b/src/adapters/in/http/openapi.rs @@ -46,8 +46,14 @@ impl Modify for SecurityAddon { http::hosts::delete_host, http::endpoints::run_endpoint, http::endpoints::run_endpoint_get, + http::endpoints::endpoint_counts, + http::credentials::list_credentials, + http::settings::get_settings, + http::host_search::search_hosts, http::endpoints::list_endpoints, http::projects::list_projects, + http::users::get_auth, + http::topology::get_topology, http::projects::sync_project_now, http::config::get_config, http::config::get_config_file, @@ -75,10 +81,36 @@ impl Modify for SecurityAddon { http::sync::SyncResult, http::enrichers::EnrichResult, http::endpoints::EndpointInfo, + http::endpoints::EndpointTransform, + http::endpoints::EndpointCounts, http::endpoints::EndpointUnavailableBody, http::enrichers::EnricherInfo, http::health::ReadyStatus, http::projects::ProjectInfo, + http::projects::ProjectUsers, + http::credentials::CredentialInfo, + http::credentials::FileKeyInfo, + http::credentials::CredentialUsersInfo, + http::settings::Settings, + http::host_search::HostSearchResult, + http::host_search::FoundHost, + http::host_search::FoundIn, + http::settings::ServerSettings, + http::settings::UiSettings, + http::settings::CacheSettings, + http::settings::ProjectsSettings, + http::settings::SecretsSettings, + http::settings::VaultSettings, + http::settings::ConfigApiSettings, + http::users::AuthInfo, + crate::domain::topology::Topology, + crate::domain::topology::Peer, + crate::domain::topology::PeerSource, + http::users::Me, + http::users::Methods, + http::users::LdapInfo, + http::users::AccessRuleInfo, + http::users::ApiKeyInfo, http::projects::ProjectSyncResult, http::config::ConfigBundle, http::config::ConfigFileInfo, @@ -96,6 +128,9 @@ impl Modify for SecurityAddon { (name = "Enrichers", description = "Post-processing enrichment of cached data"), (name = "Endpoints", description = "Output endpoints for consumers (AWX, AnsibleForms)"), (name = "Projects", description = "Git project checkouts (admin-only operational routes)"), + (name = "Credentials", description = "Configured credentials: where each one's secrets come from, whether they resolve now, and what uses it (admin-only; never a secret value)"), + (name = "Topology", description = "The federation seen from this instance: the unified-api instances it pulls from, and theirs"), + (name = "Auth", description = "Who the caller is, and the ways in: LDAP login and API keys (read-only; declared in git)"), (name = "Configuration", description = "The configuration directory over HTTP (admin-only): read it, validate a proposed change exactly as --check-config would, write it atomically, and reload it into the running process. Off unless config_api.enabled is set") ), // No explicit version: utoipa takes it from Cargo.toml (CARGO_PKG_VERSION), diff --git a/src/adapters/in/http/projects.rs b/src/adapters/in/http/projects.rs index 3700b92..425f4b2 100644 --- a/src/adapters/in/http/projects.rs +++ b/src/adapters/in/http/projects.rs @@ -27,6 +27,14 @@ pub struct ProjectInfo { /// Seconds between periodic re-pulls (absent/0 = only boot and on demand) pub sync_interval_seconds: Option, pub sync_on_boot: bool, + /// Cron expression (UTC) for re-pulls, the alternative to an interval + pub schedule: Option, + /// Seconds one clone or pull may take before it is killed + pub timeout_seconds: u64, + /// What reads its scripts from this checkout: the ids whose `project_id` + /// names it. A project nothing uses can go; one with users cannot, + /// without breaking them. + pub used_by: ProjectUsers, /// Whether anything is still managing to pull this checkout — last /// attempt, last success, last error, consecutive failures. Absent until a /// sync has run through this process (boot, interval or on demand). This @@ -36,6 +44,28 @@ pub struct ProjectInfo { pub sync_health: Option, } +#[derive(Serialize, ToSchema)] +pub struct ProjectUsers { + pub sources: Vec, + pub endpoints: Vec, + pub enrichers: Vec, +} + +// Sorted ids of everything in `items` whose project is `project`. +fn users_of<'a, T: 'a>( + items: impl IntoIterator, + project: &str, + project_of: impl Fn(&T) -> Option<&str>, +) -> Vec { + let mut ids: Vec = items + .into_iter() + .filter(|(_, item)| project_of(item) == Some(project)) + .map(|(id, _)| id.clone()) + .collect(); + ids.sort(); + ids +} + #[utoipa::path( get, path = "/api/v1/projects", @@ -72,11 +102,20 @@ pub async fn list_projects( projects.push(ProjectInfo { project_id: id.clone(), name: project.name.clone(), - git_url: project.git_url.clone(), + // Shown, so never with a token someone wrote into it; git itself + // clones from the configured value. + git_url: crate::domain::topology::without_userinfo(&project.git_url), branch: project.branch.clone(), checkout_present, sync_interval_seconds: project.sync_interval_seconds, sync_on_boot: project.sync_on_boot, + schedule: project.schedule.clone(), + timeout_seconds: project.timeout_seconds, + used_by: ProjectUsers { + sources: users_of(&config.sources, id, |s| Some(s.project_id.as_str())), + endpoints: users_of(&config.endpoints, id, |e| e.project_id.as_deref()), + enrichers: users_of(&config.enrichers, id, |e| e.project_id.as_deref()), + }, sync_health: state.project_health.get(id).map(Into::into), }); } diff --git a/src/adapters/in/http/routes.rs b/src/adapters/in/http/routes.rs index c98e882..13edc28 100644 --- a/src/adapters/in/http/routes.rs +++ b/src/adapters/in/http/routes.rs @@ -74,7 +74,19 @@ pub fn create_router(state: Arc, api_keys: Arc) -> Rou "/api/v1/endpoints/{id}", post(http::endpoints::run_endpoint).get(http::endpoints::run_endpoint_get), ) + .route( + "/api/v1/endpoints/{id}/counts", + get(http::endpoints::endpoint_counts), + ) .route("/api/v1/projects", get(http::projects::list_projects)) + .route("/api/v1/settings", get(http::settings::get_settings)) + .route("/api/v1/hosts/search", get(http::host_search::search_hosts)) + .route( + "/api/v1/credentials", + get(http::credentials::list_credentials), + ) + .route("/api/v1/auth", get(http::users::get_auth)) + .route("/api/v1/topology", get(http::topology::get_topology)) .route( "/api/v1/projects/{id}/sync", post(http::projects::sync_project_now), @@ -129,9 +141,28 @@ pub fn create_router(state: Arc, api_keys: Arc) -> Rou // read from the current snapshot), which is what lets a reload flip it. // It carries the key registry itself, since the auth middleware only // wraps the group above. + // The session store too: with metrics_require_auth on, a signed-in + // browser (the UI's Metrics and Overview pages) is let in by its session, + // as it is everywhere else. let metrics_route = Router::new() .route("/metrics", get(http::metrics::metrics)) - .layer(axum::Extension(ApiKeys(Arc::clone(&api_keys)))); + .layer(axum::Extension(ApiKeys(Arc::clone(&api_keys)))) + .layer(axum::Extension(Sessions(Arc::clone(&sessions)))); + + // The built UI, when this instance serves one (server.ui.assets_dir, and + // only with the browsable UI on: it signs in through the login page). + let ui_assets = ui + .assets_dir + .as_ref() + .filter(|_| ui.enabled) + .map(http::ui_assets::UiAssets::new); + let metrics_route = match &ui_assets { + Some(assets) => metrics_route.layer(middleware::from_fn_with_state( + assets.clone(), + http::ui_assets::page_for_browsers, + )), + None => metrics_route, + }; // Login is public by necessity: a page that asks for a credential cannot // require one. It carries the key registry and the session store itself, @@ -154,10 +185,15 @@ pub fn create_router(state: Arc, api_keys: Arc) -> Rou Router::new() }; + // "/" is the UI when there is one (the fallback below serves its + // index.html), and the API's documentation otherwise. let router = Router::new() - .route("/", get(|| async { Redirect::permanent("/swagger-ui/") })) .route("/healthz", get(http::health::healthz)) .route("/readyz", get(http::health::readyz)); + let router = match &ui_assets { + Some(_) => router, + None => router.route("/", get(|| async { Redirect::permanent("/swagger-ui/") })), + }; // The CORS and body-limit middlewares below read their reloadable // settings from the state; kept as clones because with_state consumes @@ -174,6 +210,14 @@ pub fn create_router(state: Arc, api_keys: Arc) -> Rou .config(swagger_config()), ) .with_state(state); + // Everything no route matched: the UI's files and pages (ui_assets keeps + // unmatched API paths a JSON 404). + let router = match ui_assets { + Some(assets) => { + router.fallback(axum::routing::any(http::ui_assets::serve).with_state(assets)) + } + None => router, + }; // The request body limit, from the current snapshot per request — which // is what makes server.max_body_bytes reloadable (a DefaultBodyLimit diff --git a/src/adapters/in/http/session.rs b/src/adapters/in/http/session.rs index f6a077b..b07fc46 100644 --- a/src/adapters/in/http/session.rs +++ b/src/adapters/in/http/session.rs @@ -21,14 +21,24 @@ use crate::adapters::r#in::http::auth::Permissions; // // What a session is NOT: a second credential store. Logging in means presenting // an existing API key, validated by exactly the same constant-time compare as a -// header would be. The session is a short-lived handle on that key, so the -// permission model has one source (`api_keys.yaml`) and the UI inherits it. +// header would be, or a directory account checked by the directory itself. +// The session is a short-lived handle on what that login granted. pub const COOKIE_NAME: &str = "uapi_session"; +// How a session was opened, which decides what revokes it: an API key session +// ends when its key leaves api_keys.yaml, an LDAP one when the LDAP +// configuration (the directory or its access rules) changes. +#[derive(Clone, Copy, Debug, PartialEq)] +pub enum SessionKind { + ApiKey, + Directory, +} + pub struct Session { - // Which key this session is, so it can be revoked with the key and named in - // the access log exactly as a header-authenticated request is. + // Who this session is: the key's name, or `ldap:`. Named in the + // access log exactly as a header-authenticated request is. pub key_name: String, + pub kind: SessionKind, pub permissions: Permissions, // Paired with the session and required on every unsafe request that // authenticates by cookie. See `auth::require_api_key` for why. @@ -58,7 +68,13 @@ impl SessionStore { // service has no external data dependency. The alternative — signed cookies // that survive a restart — means a key revoked while the process was down // comes back to life with it. - pub fn create(&self, key_name: String, permissions: Permissions, ttl: Duration) -> Issued { + pub fn create( + &self, + key_name: String, + kind: SessionKind, + permissions: Permissions, + ttl: Duration, + ) -> Issued { // Expired entries are normally dropped when looked up, but a browser // that logs in and is never used again is never looked up. Sweeping on // login bounds the map by "sessions in use" instead of "logins since @@ -73,6 +89,7 @@ impl SessionStore { token.clone(), Session { key_name, + kind, permissions, csrf: csrf.clone(), expires_at: Instant::now() + ttl, @@ -93,6 +110,7 @@ impl SessionStore { } else { return Some(Authenticated { key_name: session.key_name.clone(), + kind: session.kind, permissions: session.permissions.clone(), csrf: session.csrf.clone(), }); @@ -113,8 +131,17 @@ impl SessionStore { // before it — the console would outlive the revocation, which is exactly // the property a revocation is for. pub fn retain_keys(&self, live: &[String]) { + self.sessions.retain(|_, session| { + session.kind != SessionKind::ApiKey || live.contains(&session.key_name) + }); + } + + // Every LDAP session, ended. Returns how many there were, for the log. + pub fn end_directory_sessions(&self) -> usize { + let before = self.sessions.len(); self.sessions - .retain(|_, session| live.contains(&session.key_name)); + .retain(|_, session| session.kind != SessionKind::Directory); + before - self.sessions.len() } pub fn len(&self) -> usize { @@ -133,6 +160,7 @@ pub struct Issued { pub struct Authenticated { pub key_name: String, + pub kind: SessionKind, pub permissions: Permissions, pub csrf: String, } @@ -325,7 +353,12 @@ mod tests { #[test] fn a_session_authenticates_as_its_key() { let store = SessionStore::new(); - let issued = store.create("AWX".into(), admin(), Duration::from_secs(60)); + let issued = store.create( + "AWX".into(), + SessionKind::ApiKey, + admin(), + Duration::from_secs(60), + ); let found = store.get(&issued.token).expect("session"); assert_eq!(found.key_name, "AWX"); assert_eq!(found.csrf, issued.csrf); @@ -340,7 +373,12 @@ mod tests { #[test] fn an_expired_session_stops_working_and_is_dropped() { let store = SessionStore::new(); - let issued = store.create("AWX".into(), admin(), Duration::from_secs(0)); + let issued = store.create( + "AWX".into(), + SessionKind::ApiKey, + admin(), + Duration::from_secs(0), + ); assert!(store.get(&issued.token).is_none()); assert_eq!(store.len(), 0, "the expired entry must not be kept"); } @@ -348,7 +386,12 @@ mod tests { #[test] fn logout_ends_the_session() { let store = SessionStore::new(); - let issued = store.create("AWX".into(), admin(), Duration::from_secs(60)); + let issued = store.create( + "AWX".into(), + SessionKind::ApiKey, + admin(), + Duration::from_secs(60), + ); store.remove(&issued.token); assert!(store.get(&issued.token).is_none()); } @@ -358,8 +401,18 @@ mod tests { #[test] fn revoking_a_key_ends_its_sessions_only() { let store = SessionStore::new(); - let stays = store.create("AWX".into(), admin(), Duration::from_secs(60)); - let goes = store.create("Retired".into(), admin(), Duration::from_secs(60)); + let stays = store.create( + "AWX".into(), + SessionKind::ApiKey, + admin(), + Duration::from_secs(60), + ); + let goes = store.create( + "Retired".into(), + SessionKind::ApiKey, + admin(), + Duration::from_secs(60), + ); store.retain_keys(&["AWX".to_string()]); @@ -367,11 +420,50 @@ mod tests { assert!(store.get(&goes.token).is_none()); } + // An LDAP session is not a key: a key reload must leave it alone, and an + // LDAP change must end it without touching the key sessions. + #[test] + fn each_kind_of_session_is_revoked_by_its_own_change() { + let store = SessionStore::new(); + let key = store.create( + "AWX".into(), + SessionKind::ApiKey, + admin(), + Duration::from_secs(60), + ); + let person = store.create( + "ldap:jdoe".into(), + SessionKind::Directory, + admin(), + Duration::from_secs(60), + ); + + store.retain_keys(&["AWX".to_string()]); + assert!( + store.get(&person.token).is_some(), + "a key reload ended an LDAP session" + ); + + assert_eq!(store.end_directory_sessions(), 1); + assert!(store.get(&person.token).is_none()); + assert!(store.get(&key.token).is_some()); + } + #[test] fn tokens_are_unique_and_long() { let store = SessionStore::new(); - let a = store.create("k".into(), admin(), Duration::from_secs(60)); - let b = store.create("k".into(), admin(), Duration::from_secs(60)); + let a = store.create( + "k".into(), + SessionKind::ApiKey, + admin(), + Duration::from_secs(60), + ); + let b = store.create( + "k".into(), + SessionKind::ApiKey, + admin(), + Duration::from_secs(60), + ); assert_ne!(a.token, b.token); assert_ne!(a.token, a.csrf, "the CSRF token is not the session token"); assert_eq!(a.token.len(), 64); diff --git a/src/adapters/in/http/settings.rs b/src/adapters/in/http/settings.rs new file mode 100644 index 0000000..e048e83 --- /dev/null +++ b/src/adapters/in/http/settings.rs @@ -0,0 +1,184 @@ +use axum::Json; +use axum::extract::State; +use serde::Serialize; +use std::sync::Arc; +use utoipa::ToSchema; + +use crate::AppState; +use crate::adapters::r#in::http::auth::AuthContext; +use crate::adapters::r#in::http::error::{ApiError, ErrorBody}; + +// The configuration in force, read-only, for the UI's Configuration page. +// +// Not the file: the VALUES, defaults included. config.yaml is short because +// most settings are left at their default, and the question an operator +// brings ("is /metrics public here?", "how long does a session last?") is +// usually about one of those. The file itself, when it is wanted, is +// GET /api/v1/config/config.yaml (with the configuration API on). +// +// Grouped as config.yaml is, and named by the file's keys. Admin-only, like +// the configuration API: the listen address, the Vault and the paths are a +// map of the deployment. Never a secret: Vault is described by its address, +// mount and how it authenticates, not by a token. + +#[derive(Serialize, ToSchema)] +pub struct Settings { + /// The unified-api version running + pub version: &'static str, + /// How long this process has been running + pub uptime_seconds: u64, + pub server: ServerSettings, + pub cache: CacheSettings, + pub projects: ProjectsSettings, + pub secrets: SecretsSettings, + pub config_api: ConfigApiSettings, + /// Keys a reload has changed in the files that this process cannot adopt + /// without a restart; empty when none + pub restart_pending: Vec, +} + +#[derive(Serialize, ToSchema)] +pub struct ServerSettings { + /// Restart-only; null when the process was not built from a config file + pub host: Option, + /// Restart-only, like host + pub port: Option, + pub cors_allowed_origins: Vec, + pub readyz_require_all_sources: bool, + pub metrics_require_auth: bool, + pub refresh_timeout_seconds: u64, + pub refresh_max_concurrent: usize, + pub shutdown_grace_seconds: u64, + pub max_body_bytes: usize, + pub ui: UiSettings, +} + +#[derive(Serialize, ToSchema)] +pub struct UiSettings { + pub enabled: bool, + pub session_ttl_seconds: u64, + /// Whether directory (LDAP) login is configured; its details are on + /// GET /api/v1/auth + pub ldap: bool, +} + +#[derive(Serialize, ToSchema)] +pub struct CacheSettings { + /// Where the cache snapshot is written; null = no snapshot (restart-only) + pub persistence_path: Option, + pub persistence_interval_seconds: Option, + /// When a snapshot was last written; null before the first, or with + /// persistence off. An idle cache skips its snapshots on purpose, so + /// this ages while nothing changes. + pub last_saved_age_seconds: Option, + /// Snapshot writes that have failed since the last one that worked + pub consecutive_failures: u32, +} + +#[derive(Serialize, ToSchema)] +pub struct ProjectsSettings { + /// Where project checkouts live (restart-only) + pub dir: Option, +} + +#[derive(Serialize, ToSchema)] +pub struct SecretsSettings { + /// How long a resolved secret is reused: also how long a rotation takes + /// to be picked up + pub cache_ttl_seconds: u64, + pub vault: Option, +} + +#[derive(Serialize, ToSchema)] +pub struct VaultSettings { + pub address: String, + pub mount: String, + /// `kubernetes` (with its role) or `token` (with the env var holding it) + pub auth: &'static str, + pub kubernetes_role: Option, + /// The environment variable the token is read from, for token auth + pub token_env: Option, + pub timeout_seconds: u64, +} + +#[derive(Serialize, ToSchema)] +pub struct ConfigApiSettings { + /// Whether the configuration directory is readable and writable over + /// /api/v1/config (restart-only) + pub enabled: Option, +} + +#[utoipa::path( + get, + path = "/api/v1/settings", + tag = "Configuration", + responses( + (status = 200, description = "The configuration in force, defaults included, grouped and named as config.yaml is. Never a secret", body = Settings), + (status = 403, description = "API key is not admin", body = ErrorBody) + ) +)] +pub async fn get_settings( + State(state): State>, + axum::Extension(auth): axum::Extension, +) -> Result, ApiError> { + if !auth.permissions.is_admin() { + return Err(ApiError::admin_only()); + } + let config = state.config(); + let boot = state.live_settings(); + + let snapshot = state + .snapshot_health + .get(crate::adapters::out::cache::persistence::SNAPSHOT_HEALTH_KEY); + + Ok(Json(Settings { + version: env!("CARGO_PKG_VERSION"), + uptime_seconds: state.started.elapsed().as_secs(), + server: ServerSettings { + host: boot.as_ref().map(|b| b.host.clone()), + port: boot.as_ref().map(|b| b.port), + cors_allowed_origins: config.cors_allowed_origins.clone(), + readyz_require_all_sources: config.readyz_require_all_sources, + metrics_require_auth: config.metrics_require_auth, + refresh_timeout_seconds: config.refresh_timeout_seconds, + refresh_max_concurrent: config.refresh_max_concurrent, + shutdown_grace_seconds: config.shutdown_grace_seconds, + max_body_bytes: config.max_body_bytes, + ui: UiSettings { + enabled: state.ui.enabled, + session_ttl_seconds: state.ui.session_ttl_seconds, + ldap: config.ldap.is_some(), + }, + }, + cache: CacheSettings { + persistence_path: boot.as_ref().and_then(|b| b.persistence_path.clone()), + persistence_interval_seconds: boot + .as_ref() + .and_then(|b| b.persistence_interval_seconds), + last_saved_age_seconds: snapshot.as_ref().and_then(|h| h.last_success_age_seconds), + consecutive_failures: snapshot.as_ref().map_or(0, |h| h.consecutive_failures), + }, + projects: ProjectsSettings { + dir: boot.as_ref().map(|b| b.projects_dir.clone()), + }, + secrets: SecretsSettings { + cache_ttl_seconds: config.secrets.cache_ttl_seconds, + vault: config.secrets.vault.as_ref().map(|v| VaultSettings { + address: v.address.clone(), + mount: v.mount.clone(), + auth: if v.kubernetes_role.is_some() { + "kubernetes" + } else { + "token" + }, + kubernetes_role: v.kubernetes_role.clone(), + token_env: v.kubernetes_role.is_none().then(|| v.token_env.clone()), + timeout_seconds: v.timeout_seconds, + }), + }, + config_api: ConfigApiSettings { + enabled: boot.as_ref().map(|b| b.config_api_enabled), + }, + restart_pending: state.restart_pending(), + })) +} diff --git a/src/adapters/in/http/topology.rs b/src/adapters/in/http/topology.rs new file mode 100644 index 0000000..17e360b --- /dev/null +++ b/src/adapters/in/http/topology.rs @@ -0,0 +1,137 @@ +use axum::Json; +use axum::extract::{Query, State}; +use serde::Deserialize; +use std::sync::Arc; +use utoipa::IntoParams; + +use crate::AppState; +use crate::adapters::r#in::http::auth::AuthContext; +use crate::adapters::r#in::http::error::ApiError; +use crate::adapters::r#in::http::error::ErrorBody; +use crate::application::topology::{Inputs, MAX_DEPTH, describe}; +use crate::domain::topology::Topology; + +// Declared as query parameters here: the handler takes them wrapped in a +// Result (to answer a malformed one as JSON), which hides the Query the +// OpenAPI derive would otherwise infer them from. +#[derive(Deserialize, IntoParams)] +#[into_params(parameter_in = Query)] +pub struct TopologyQuery { + /// How many levels of peers to ask (default 2: this instance's peers and + /// theirs). Capped at 3; 0 maps the peers without asking them. + pub depth: Option, +} + +// The federation seen from here, for the UI's Topology page. +// +// Any key may ask, and each sees the peers behind the sources it may read: +// an edge answers its central's restricted key with what that key is for, +// and nothing about sources it cannot see. +#[utoipa::path( + get, + path = "/api/v1/topology", + tag = "Topology", + params(TopologyQuery), + responses( + (status = 200, description = "This instance, the instances it pulls from through remote sources, and theirs, as far as `depth` reaches. A peer that could not be asked carries `error` instead of `instance`. A restricted key sees each peer's version only, not the peer's own peers", body = Topology), + (status = 400, description = "`depth` is not a number from 0 to 255", body = ErrorBody), + (status = 401, description = "Not authenticated", body = ErrorBody) + ) +)] +pub async fn get_topology( + State(state): State>, + axum::Extension(auth): axum::Extension, + query: Result, axum::extract::rejection::QueryRejection>, +) -> Result, ApiError> { + // A malformed depth answers like every other failure: {"error": ...}. + let Query(query) = query.map_err(|e| ApiError::bad_request(e.body_text()))?; + let config = state.config(); + let topology = describe( + Inputs { + sources: &config.sources, + cache: state.cache.as_ref(), + health: state.sync_health.as_ref(), + secrets: state.secrets.as_ref(), + peers: Arc::clone(&state.peers), + }, + |id| auth.permissions.allows_source(id), + query.depth.unwrap_or(2).min(MAX_DEPTH), + ) + .await; + Ok(Json(if auth.permissions.is_admin() { + topology + } else { + for_restricted(topology) + })) +} + +// What a restricted caller may see of the peers behind the sources it reads. +// The peers were asked with THIS instance's key, which may be an admin key +// over there: their own peers, URLs and counts are that key's view, not the +// caller's, so only each peer's version is kept. And why a peer was not +// asked stays generic: the resolver's message names env vars and Vault +// paths, which /credentials keeps for admins. +fn for_restricted(mut topology: Topology) -> Topology { + for peer in &mut topology.peers { + if let Some(instance) = peer.instance.as_mut() { + instance.sources = None; + instance.peers = Vec::new(); + } + if peer + .error + .as_deref() + .is_some_and(|e| e.starts_with("not asked:") && !e.contains("depth limit")) + { + peer.error = Some("not asked: this instance cannot authenticate to it".to_string()); + } + } + topology +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::domain::topology::Peer; + + fn peer(instance: Option, error: Option<&str>) -> Peer { + Peer { + url: "https://edge.example".to_string(), + sources: Vec::new(), + instance: instance.map(Box::new), + error: error.map(str::to_string), + } + } + + #[test] + fn a_restricted_caller_sees_a_peers_version_and_no_secret_locations() { + let behind = Topology { + version: "0.33.0".to_string(), + sources: Some(12), + peers: vec![peer(None, None)], + }; + let topology = Topology { + version: "0.34.0".to_string(), + sources: Some(1), + peers: vec![ + peer(Some(behind), None), + peer(None, Some("not asked: Env var 'EDGE_TOKEN' not set")), + peer(None, Some("not asked: depth limit reached")), + ], + }; + let seen = for_restricted(topology); + let instance = seen.peers[0].instance.as_ref().unwrap(); + assert_eq!(instance.version, "0.33.0"); + assert!(instance.peers.is_empty() && instance.sources.is_none()); + assert!( + !seen.peers[1] + .error + .as_deref() + .unwrap() + .contains("EDGE_TOKEN") + ); + assert_eq!( + seen.peers[2].error.as_deref(), + Some("not asked: depth limit reached") + ); + } +} diff --git a/src/adapters/in/http/ui_assets.rs b/src/adapters/in/http/ui_assets.rs new file mode 100644 index 0000000..3635f39 --- /dev/null +++ b/src/adapters/in/http/ui_assets.rs @@ -0,0 +1,148 @@ +// The built unified-ui, served at "/" when `server.ui.assets_dir` names one: +// the UI and the API on one address, so the UI's calls are same-origin and the +// login's session cookie covers both, with no proxy in front. +// +// Three kinds of path reach this fallback (everything the router does not +// match): +// - a file of the build (/assets/index-3f2a.js, /favicon.svg): served as is; +// - a page of the app (/sources, /topology/...): no such file, so the app's +// index.html, whose own router draws the page; +// - an API path nothing matched (/api/v1/typo): the API's own JSON 404, never +// the app, or a client would read a 200 HTML page as an answer. + +use std::path::PathBuf; + +use axum::body::Body; +use axum::extract::{Request, State}; +use axum::http::{HeaderValue, Method, header}; +use axum::middleware::Next; +use axum::response::{IntoResponse, Response}; +use tower::ServiceExt; +use tower_http::services::{ServeDir, ServeFile}; + +use crate::adapters::r#in::http::error::ApiError; + +#[derive(Clone)] +pub struct UiAssets { + dir: PathBuf, +} + +impl UiAssets { + pub fn new(dir: impl Into) -> Self { + let dir = dir.into(); + // Not fatal: the API works without its UI. Said at startup, where a + // wrong path in a deployment is noticed, rather than as a 404 later. + if !dir.join("index.html").is_file() { + tracing::warn!( + dir = %dir.display(), + "server.ui.assets_dir has no index.html: the UI will not load" + ); + } + Self { dir } + } + + fn index(&self) -> PathBuf { + self.dir.join("index.html") + } +} + +// Paths that belong to the API: a miss there is the API's 404, not the app. +fn is_api_path(path: &str) -> bool { + ["/api", "/swagger-ui", "/api-docs"] + .iter() + .any(|prefix| path == *prefix || path.starts_with(&format!("{prefix}/"))) +} + +pub async fn serve(State(assets): State, request: Request) -> Response { + let path = request.uri().path().to_string(); + if is_api_path(&path) { + return ApiError::not_found(format!("no route for {path}")).into_response(); + } + if request.method() != Method::GET && request.method() != Method::HEAD { + return ApiError::new( + axum::http::StatusCode::METHOD_NOT_ALLOWED, + "the UI's files are read-only", + ) + .into_response(); + } + + let service = ServeDir::new(&assets.dir).fallback(ServeFile::new(assets.index())); + let response = match service.oneshot(request).await { + Ok(response) => response.map(Body::new), + Err(never) => match never {}, + }; + with_cache_policy(&path, response) +} + +// Vite names every built file after its content (index-3f2a.js), so a file +// under /assets never changes and can be kept for good; everything else (the +// index.html that names those files) must be asked again on every load, or a +// browser would keep running the previous release's app. +fn with_cache_policy(path: &str, mut response: Response) -> Response { + let policy = if path.starts_with("/assets/") && response.status().is_success() { + "public, max-age=31536000, immutable" + } else { + "no-cache" + }; + response + .headers_mut() + .insert(header::CACHE_CONTROL, HeaderValue::from_static(policy)); + response +} + +// /metrics is two things on one path: Prometheus's endpoint and the UI's +// Metrics page. A browser navigating there asks for HTML and gets the app; +// a scraper, or the page's own fetch, never does and gets the metrics. +pub async fn page_for_browsers( + State(assets): State, + request: Request, + next: Next, +) -> Response { + let wants_html = request + .headers() + .get(header::ACCEPT) + .and_then(|v| v.to_str().ok()) + .is_some_and(|accept| accept.contains("text/html")); + if !wants_html { + return next.run(request).await; + } + let response = match ServeFile::new(assets.index()).oneshot(request).await { + Ok(response) => response.map(Body::new), + Err(never) => match never {}, + }; + with_cache_policy("/metrics", response) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn api_paths_are_never_the_app() { + for path in [ + "/api", + "/api/v1/nope", + "/swagger-ui/", + "/api-docs/openapi.json", + ] { + assert!(is_api_path(path), "{path}"); + } + for path in ["/", "/sources", "/apis", "/assets/index.js", "/metrics"] { + assert!(!is_api_path(path), "{path}"); + } + } + + #[test] + fn only_built_files_are_cached_for_good() { + let ok = || Response::new(Body::empty()); + let cache = |r: Response| { + r.headers()[header::CACHE_CONTROL] + .to_str() + .unwrap() + .to_string() + }; + assert!(cache(with_cache_policy("/assets/index-3f2a.js", ok())).contains("immutable")); + assert_eq!(cache(with_cache_policy("/sources", ok())), "no-cache"); + assert_eq!(cache(with_cache_policy("/", ok())), "no-cache"); + } +} diff --git a/src/adapters/in/http/users.rs b/src/adapters/in/http/users.rs new file mode 100644 index 0000000..dd693b7 --- /dev/null +++ b/src/adapters/in/http/users.rs @@ -0,0 +1,198 @@ +use axum::Json; +use axum::extract::State; +use serde::Serialize; +use std::sync::Arc; +use utoipa::ToSchema; + +use crate::AppState; +use crate::adapters::r#in::http::auth::{ + ApiKeys, AuthContext, AuthMethod, BrowserIdentity, Permissions, +}; +use crate::adapters::r#in::http::error::ErrorBody; +use crate::domain::api_key::ApiKeyRole; + +// Who can get in, and how — read-only, for the UI's Users page. +// +// Read-only by design: users and the directory are declared in git +// (config.yaml `server.ui.ldap`, api_keys.yaml) and pushed like the rest of +// the configuration, so this route describes what is in force rather than +// offering a second place to change it. Everyone authenticated sees who THEY +// are; only an admin sees the rest, because the access rules and key scopes +// are a map of who holds what. + +#[derive(Serialize, ToSchema)] +pub struct AuthInfo { + pub me: Me, + pub methods: Methods, + /// The LDAP configuration and its access rules. Admin only; absent when + /// LDAP login is not configured. + #[serde(skip_serializing_if = "Option::is_none")] + pub ldap: Option, + /// The API keys in force, by name and scope — never a secret. Admin only. + #[serde(skip_serializing_if = "Option::is_none")] + pub api_keys: Option>, +} + +#[derive(Serialize, ToSchema)] +pub struct Me { + /// The key's name, or `ldap:`. Null on an open instance. + pub name: Option, + /// How this request authenticated: `open`, `api_key` or `ldap` + pub method: &'static str, + pub role: &'static str, + /// Source ids a restricted caller may read and sync (empty for admin) + pub sources: Vec, + /// Endpoint ids a restricted caller may run (empty for admin) + pub endpoints: Vec, + /// The token a browser session sends as `X-CSRF-Token` on every write. + /// Only for a session (cookie) caller; a header-authenticated caller + /// needs none. Safe to return: another site cannot read this response. + #[serde(skip_serializing_if = "Option::is_none")] + pub csrf_token: Option, +} + +#[derive(Serialize, ToSchema)] +pub struct Methods { + /// API keys are always accepted, as a header or through the login form + pub api_key: bool, + /// Whether the login form offers a directory account + pub ldap: bool, +} + +#[derive(Serialize, ToSchema)] +pub struct LdapInfo { + pub url: String, + pub starttls: bool, + pub ca_file: Option, + /// credentials.yaml id holding the service account (its secret is not shown) + pub bind_credential: String, + pub search_base: String, + pub user_filter: String, + pub group_attribute: String, + pub timeout_seconds: u64, + pub access: Vec, +} + +#[derive(Serialize, ToSchema)] +pub struct AccessRuleInfo { + pub name: String, + pub groups: Vec, + pub users: Vec, + pub role: &'static str, + pub sources: Vec, + pub endpoints: Vec, +} + +#[derive(Serialize, ToSchema)] +pub struct ApiKeyInfo { + pub name: String, + pub role: &'static str, + pub sources: Vec, + pub endpoints: Vec, +} + +fn role_name(role: ApiKeyRole) -> &'static str { + match role { + ApiKeyRole::Admin => "admin", + ApiKeyRole::Restricted => "restricted", + } +} + +// Sorted so the page and any diff of this response are stable. +fn scope(permissions: &Permissions) -> (&'static str, Vec, Vec) { + match permissions { + Permissions::Admin => ("admin", Vec::new(), Vec::new()), + Permissions::Scoped { sources, endpoints } => { + let mut sources: Vec = sources.iter().cloned().collect(); + let mut endpoints: Vec = endpoints.iter().cloned().collect(); + sources.sort(); + endpoints.sort(); + ("restricted", sources, endpoints) + } + } +} + +#[utoipa::path( + get, + path = "/api/v1/auth", + tag = "Auth", + responses( + (status = 200, description = "Who the caller is, and — for an admin — every way in: the LDAP configuration with its access rules and the API keys by name", body = AuthInfo), + (status = 401, description = "Not authenticated", body = ErrorBody) + ) +)] +pub async fn get_auth( + State(state): State>, + axum::Extension(auth): axum::Extension, + axum::Extension(keys): axum::Extension, + session: Option>, +) -> Json { + let config = state.config(); + let (role, sources, endpoints) = scope(&auth.permissions); + let admin = auth.permissions.is_admin(); + + let me = Me { + name: auth.key_name.clone(), + method: match auth.method { + AuthMethod::Open => "open", + AuthMethod::ApiKey => "api_key", + AuthMethod::Ldap => "ldap", + }, + role, + sources, + endpoints, + csrf_token: session.map(|axum::Extension(identity)| identity.csrf), + }; + + let ldap = config.ldap.as_ref().filter(|_| admin).map(|ldap| LdapInfo { + url: ldap.url.clone(), + starttls: ldap.starttls, + ca_file: ldap.ca_file.clone(), + bind_credential: ldap.bind_credential.clone(), + search_base: ldap.search_base.clone(), + user_filter: ldap.user_filter.clone(), + group_attribute: ldap.group_attribute.clone(), + timeout_seconds: ldap.timeout_seconds, + access: ldap + .access + .iter() + .map(|rule| AccessRuleInfo { + name: rule.name.clone(), + groups: rule.groups.clone(), + users: rule.users.clone(), + role: role_name(rule.role), + sources: rule.sources.clone(), + endpoints: rule.endpoints.clone(), + }) + .collect(), + }); + + let api_keys = admin.then(|| { + let mut list: Vec = keys + .0 + .load() + .iter() + .map(|key| { + let (role, sources, endpoints) = scope(&key.permissions); + ApiKeyInfo { + name: key.name.clone(), + role, + sources, + endpoints, + } + }) + .collect(); + list.sort_by(|a, b| a.name.cmp(&b.name)); + list + }); + + Json(AuthInfo { + me, + methods: Methods { + api_key: true, + ldap: config.ldap.is_some(), + }, + ldap, + api_keys, + }) +} diff --git a/src/adapters/in/scheduler/mod.rs b/src/adapters/in/scheduler/mod.rs index 62d66af..e799995 100644 --- a/src/adapters/in/scheduler/mod.rs +++ b/src/adapters/in/scheduler/mod.rs @@ -1417,6 +1417,8 @@ mod tests { shutdown_grace_seconds: previous.shutdown_grace_seconds, refresh_timeout_seconds: previous.refresh_timeout_seconds, refresh_max_concurrent: previous.refresh_max_concurrent, + ldap: previous.ldap.clone(), + ui_home: previous.ui_home.clone(), })); state.reload.bump(); diff --git a/src/adapters/out/directory/ldap.rs b/src/adapters/out/directory/ldap.rs new file mode 100644 index 0000000..b8a2aea --- /dev/null +++ b/src/adapters/out/directory/ldap.rs @@ -0,0 +1,223 @@ +use std::sync::Arc; +use std::time::Duration; + +use ldap3::{LdapConnAsync, LdapConnSettings, Scope, SearchEntry}; + +use crate::domain::ldap::user_filter; +use crate::ports::directory::{ + DirectoryError, DirectoryFuture, DirectoryLogin, DirectoryPort, DirectoryUser, +}; + +// LDAP result code for a wrong password or an unknown bind DN. +const INVALID_CREDENTIALS: u32 = 49; + +// DirectoryPort → an LDAP server, with the usual search-then-bind: +// +// 1. bind as the service account (it may read users; a person may not +// know their own DN), +// 2. search for exactly one entry matching the user filter, +// 3. bind AS that entry with the password typed: the directory itself +// checks it, so no password or hash ever needs to be readable here. +// +// One connection per login, closed straight after. Logins are rare and a +// pooled connection would sit bound as the last person who logged in. +pub struct LdapDirectory; + +impl DirectoryPort for LdapDirectory { + fn authenticate<'a>(&'a self, request: DirectoryLogin<'a>) -> DirectoryFuture<'a> { + Box::pin(authenticate(request)) + } +} + +async fn authenticate(request: DirectoryLogin<'_>) -> Result { + // An empty password is not a wrong password to LDAP: a simple bind with + // a DN and no password is an "unauthenticated bind", which many servers, + // Active Directory included, answer with SUCCESS. Without this check an + // empty password field logs anyone in as anyone. + if request.password.is_empty() || request.username.trim().is_empty() { + return Err(DirectoryError::InvalidCredentials); + } + + let config = request.config; + let timeout = Duration::from_secs(config.timeout_seconds); + let unavailable = |what: &str, e: ldap3::LdapError| { + DirectoryError::Unavailable(format!("{what} ({}): {e}", config.url)) + }; + + let mut settings = LdapConnSettings::new() + .set_conn_timeout(timeout) + .set_starttls(config.starttls); + if let Some(ca_file) = &config.ca_file { + settings = settings.set_config(tls_config(ca_file).map_err(DirectoryError::Unavailable)?); + } + + let (conn, mut ldap) = LdapConnAsync::with_settings(settings, &config.url) + .await + .map_err(|e| unavailable("connect", e))?; + // The connection is driven by its own task; it ends when `ldap` is dropped. + ldap3::drive!(conn); + + ldap.with_timeout(timeout) + .simple_bind(request.bind_dn, request.bind_password) + .await + .and_then(|result| result.success()) + .map_err(|e| unavailable("service account bind", e))?; + + let filter = user_filter(&config.user_filter, request.username); + let (entries, _) = ldap + .with_timeout(timeout) + .search( + &config.search_base, + Scope::Subtree, + &filter, + vec![ + config.group_attribute.as_str(), + config.username_attribute.as_str(), + ], + ) + .await + .and_then(|result| result.success()) + .map_err(|e| unavailable("user search", e))?; + + // Zero is an unknown user; more than one is a filter too loose to say + // WHO is logging in, and picking the first would be a guess. + let [entry] = entries.as_slice() else { + let _ = ldap.unbind().await; + if entries.len() > 1 { + tracing::warn!( + matches = entries.len(), + "LDAP user filter matched more than one entry; refusing the login" + ); + } + return Err(DirectoryError::InvalidCredentials); + }; + let entry = SearchEntry::construct(entry.clone()); + + let user_bind = ldap + .with_timeout(timeout) + .simple_bind(&entry.dn, request.password) + .await + .map_err(|e| unavailable("user bind", e))?; + let _ = ldap.unbind().await; + + match user_bind.rc { + 0 => {} + INVALID_CREDENTIALS => return Err(DirectoryError::InvalidCredentials), + rc => { + return Err(DirectoryError::Unavailable(format!( + "user bind ({}): result code {rc} {}", + config.url, user_bind.text + ))); + } + } + + // Attribute names are case-insensitive in LDAP, and servers echo them + // back in whatever case they store. + let attribute = |wanted: &str| { + entry + .attrs + .iter() + .find(|(name, _)| name.eq_ignore_ascii_case(wanted)) + .map(|(_, values)| values.clone()) + .unwrap_or_default() + }; + let member_of = attribute(&config.group_attribute); + let username = attribute(&config.username_attribute).into_iter().next(); + + Ok(DirectoryUser { + dn: entry.dn, + username, + member_of, + }) +} + +// A TLS configuration that trusts exactly the CAs in `ca_file`, for a +// directory signed by a private CA. Built per login: the file is small, and +// reading it each time means a rotated CA needs no restart. +fn tls_config(ca_file: &str) -> Result, String> { + use rustls_pki_types::CertificateDer; + use rustls_pki_types::pem::PemObject; + + let mut roots = rustls::RootCertStore::empty(); + let certs = CertificateDer::pem_file_iter(ca_file) + .map_err(|e| format!("ldap.ca_file '{ca_file}': {e}"))?; + for cert in certs { + let cert = cert.map_err(|e| format!("ldap.ca_file '{ca_file}': {e}"))?; + roots + .add(cert) + .map_err(|e| format!("ldap.ca_file '{ca_file}': {e}"))?; + } + if roots.is_empty() { + return Err(format!("ldap.ca_file '{ca_file}' holds no certificate")); + } + + let config = rustls::ClientConfig::builder_with_provider(Arc::new( + rustls::crypto::ring::default_provider(), + )) + .with_safe_default_protocol_versions() + .map_err(|e| format!("TLS setup: {e}"))? + .with_root_certificates(roots) + .with_no_client_auth(); + Ok(Arc::new(config)) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::domain::ldap::LdapConfig; + + fn config() -> LdapConfig { + LdapConfig { + // Nothing listens here: any attempt to connect fails, which is + // what these tests rely on to prove the guard ran first. + url: "ldap://127.0.0.1:1".to_string(), + starttls: false, + ca_file: None, + bind_credential: "cred-ldap".to_string(), + search_base: "DC=example,DC=com".to_string(), + user_filter: "(sAMAccountName={username})".to_string(), + group_attribute: "memberOf".to_string(), + username_attribute: "sAMAccountName".to_string(), + timeout_seconds: 1, + access: Vec::new(), + } + } + + // The unauthenticated-bind trap: refused before any connection is made, + // so the answer cannot depend on how the server treats it. + #[tokio::test] + async fn an_empty_password_never_reaches_the_directory() { + let config = config(); + let result = LdapDirectory + .authenticate(DirectoryLogin { + config: &config, + bind_dn: "CN=svc", + bind_password: "x", + username: "jdoe", + password: "", + }) + .await; + assert_eq!(result.unwrap_err(), DirectoryError::InvalidCredentials); + } + + #[tokio::test] + async fn an_unreachable_directory_is_unavailable_not_a_wrong_password() { + let config = config(); + let result = LdapDirectory + .authenticate(DirectoryLogin { + config: &config, + bind_dn: "CN=svc", + bind_password: "x", + username: "jdoe", + password: "secret", + }) + .await; + assert!(matches!(result, Err(DirectoryError::Unavailable(_)))); + } + + #[test] + fn a_missing_ca_file_is_named() { + let err = tls_config("/nonexistent/ca.pem").unwrap_err(); + assert!(err.contains("/nonexistent/ca.pem"), "{err}"); + } +} diff --git a/src/adapters/out/directory/mod.rs b/src/adapters/out/directory/mod.rs new file mode 100644 index 0000000..d599570 --- /dev/null +++ b/src/adapters/out/directory/mod.rs @@ -0,0 +1 @@ +pub mod ldap; diff --git a/src/adapters/out/mod.rs b/src/adapters/out/mod.rs index 75872c2..15650dc 100644 --- a/src/adapters/out/mod.rs +++ b/src/adapters/out/mod.rs @@ -3,9 +3,11 @@ pub mod cache; pub mod config; pub mod connectors; +pub mod directory; pub mod enrichers; pub mod git; pub mod output; +pub mod peers; // Not a port implementation: the scrubbed-environment Command builder shared // by every adapter that spawns a local script. pub(crate) mod process_env; diff --git a/src/adapters/out/peers/http.rs b/src/adapters/out/peers/http.rs new file mode 100644 index 0000000..4f957b8 --- /dev/null +++ b/src/adapters/out/peers/http.rs @@ -0,0 +1,174 @@ +use std::time::Duration; + +use serde::Deserialize; + +use crate::domain::topology::Topology; +use crate::ports::peer::{PeerFuture, PeerPort, PeerRequest}; + +// PeerPort → the peer's own HTTP API, with the key the remote sources use. +// +// GET /api/v1/topology first. A peer older than that route answers 404; its +// index (GET /api/v1/, readable by any key) still says which version it +// runs, so the map shows it as present but unable to describe its peers. +// +// One client per TLS policy, kept for the life of the process rather than +// built per call: a client owns the connection pool, so a fresh one per +// call meant a fresh DNS lookup, TCP connect and TLS handshake to every peer +// on every page load. Kept, the connections stay open between loads (the +// page refreshes every minute, inside the pool's idle window), and a slow +// resolver is paid once rather than every time. +pub struct HttpPeers { + verified: reqwest::Client, + insecure: reqwest::Client, +} + +impl HttpPeers { + pub fn new() -> Self { + let build = |insecure: bool| { + reqwest::Client::builder() + // Same opt-in as the remote connector's: self-signed edges exist + .danger_accept_invalid_certs(insecure) + .build() + .expect("a client with no custom TLS roots cannot fail to build") + }; + HttpPeers { + verified: build(false), + insecure: build(true), + } + } +} + +impl Default for HttpPeers { + fn default() -> Self { + Self::new() + } +} + +impl PeerPort for HttpPeers { + fn describe(&self, request: PeerRequest) -> PeerFuture { + let client = if request.insecure_tls { + self.insecure.clone() + } else { + self.verified.clone() + }; + Box::pin(describe(client, request)) + } +} + +#[derive(Deserialize)] +struct Index { + version: String, +} + +// The client is a cheap handle onto the shared pool (an Arc inside), so +// cloning it per call keeps the connections. The timeout is per request, +// since it is the caller's (the page's) bound, not the pool's. +// The most a peer's answer may weigh. A topology is a few kilobytes; a peer +// that sends megabytes is broken or hostile, and is not read into memory. +const MAX_BODY_BYTES: usize = 4 * 1024 * 1024; + +// One budget for the whole exchange, the 404 fallback included: an old peer +// must not take twice the time the page was promised. +async fn describe(client: reqwest::Client, request: PeerRequest) -> Result { + let timeout = Duration::from_secs(request.timeout_seconds); + match tokio::time::timeout(timeout, ask(&client, &request)).await { + Ok(answer) => answer, + Err(_) => Err("did not answer in time".to_string()), + } +} + +// The body, read a chunk at a time and refused past MAX_BODY_BYTES, then +// parsed: `.json()` would read any size whole first. +async fn read_json( + mut response: reqwest::Response, + what: &str, +) -> Result { + let mut body = Vec::new(); + while let Some(chunk) = response + .chunk() + .await + .map_err(|e| format!("unreadable {what}: {e}"))? + { + if body.len() + chunk.len() > MAX_BODY_BYTES { + return Err(format!( + "{what} larger than {} MB", + MAX_BODY_BYTES / 1024 / 1024 + )); + } + body.extend_from_slice(&chunk); + } + serde_json::from_slice(&body).map_err(|e| format!("unreadable {what}: {e}")) +} + +async fn ask(client: &reqwest::Client, request: &PeerRequest) -> Result { + let get = |path: String| { + let mut call = client.get(format!("{}{path}", request.url)); + if let Some(key) = &request.api_key { + call = call.header("x-api-key", key); + } + call.send() + }; + + let response = get(format!("/api/v1/topology?depth={}", request.depth)) + .await + .map_err(|e| unreachable_message(&e))?; + + match response.status().as_u16() { + 200 => read_json::(response, "topology").await, + 404 => { + let index = get("/api/v1/".to_string()) + .await + .map_err(|e| unreachable_message(&e))?; + if !index.status().is_success() { + return Err(format!("answered HTTP {}", index.status().as_u16())); + } + let index: Index = read_json(index, "index").await?; + Ok(Topology { + version: index.version, + sources: None, + peers: Vec::new(), + }) + } + 401 => Err("refused the key (401)".to_string()), + 403 => Err("the key may not describe it (403)".to_string()), + other => Err(format!("answered HTTP {other}")), + } +} + +// reqwest's error text for a timeout or a refused connection is long and +// nested; the map needs the gist. +fn unreachable_message(error: &reqwest::Error) -> String { + if error.is_timeout() { + "did not answer in time".to_string() + } else if error.is_connect() { + "unreachable".to_string() + } else { + format!("request failed: {error}") + } +} + +#[cfg(test)] +mod tests { + use super::*; + + // Whether a dead address refuses at once or swallows the packets until + // the timeout depends on the network (a WSL host does the latter); either + // way the map must say the peer cannot be reached, not invent a status. + #[tokio::test] + async fn a_dead_address_reads_as_unreachable() { + let result = HttpPeers::new() + .describe(PeerRequest { + url: "http://127.0.0.1:1".to_string(), + api_key: None, + insecure_tls: false, + timeout_seconds: 1, + depth: 1, + }) + .await; + let reason = result.unwrap_err(); + assert!( + reason == "unreachable" || reason == "did not answer in time", + "got: {reason}" + ); + } +} diff --git a/src/adapters/out/peers/mod.rs b/src/adapters/out/peers/mod.rs new file mode 100644 index 0000000..3883215 --- /dev/null +++ b/src/adapters/out/peers/mod.rs @@ -0,0 +1 @@ +pub mod http; diff --git a/src/adapters/out/secrets/mock.rs b/src/adapters/out/secrets/mock.rs index aa61bbd..d85043c 100644 --- a/src/adapters/out/secrets/mock.rs +++ b/src/adapters/out/secrets/mock.rs @@ -18,6 +18,18 @@ impl MockSecrets { secrets: HashMap::new(), } } + + // A credential the double resolves, e.g. the LDAP service account. + pub fn with(mut self, credential_id: &str, fields: [(&str, &str); N]) -> Self { + self.secrets.insert( + credential_id.to_string(), + fields + .into_iter() + .map(|(key, value)| (key.to_string(), value.to_string())) + .collect(), + ); + self + } } impl Default for MockSecrets { diff --git a/src/application/config.rs b/src/application/config.rs index 528d00e..2918beb 100644 --- a/src/application/config.rs +++ b/src/application/config.rs @@ -117,6 +117,21 @@ pub fn apply(state: &AppState, cfg: &AppConfig) -> ReloadReport { .resize(next.refresh_max_concurrent, next.refresh_timeout_seconds); } + // A session carries the permissions its rules granted at login. Changed + // rules (or a different directory) must not leave people working under + // the old ones until their session expires: those sessions end, and the + // next login is judged by the new rules. API key sessions are not + // touched here; the key swap revokes those. + if previous.ldap != next.ldap { + let ended = state.sessions.end_directory_sessions(); + if ended > 0 { + info!( + ended, + "LDAP configuration changed; its browser sessions ended" + ); + } + } + let mut report = diff(&previous, &next, cfg, state.live_settings()); // Remember what this reload could NOT adopt, so the scrape-time gauge @@ -156,6 +171,8 @@ fn runtime_config(cfg: &AppConfig) -> RuntimeConfig { shutdown_grace_seconds: cfg.server.shutdown_grace_seconds, refresh_timeout_seconds: cfg.server.refresh_timeout_seconds, refresh_max_concurrent: cfg.server.refresh_max_concurrent, + ldap: cfg.server.ui.ldap.clone(), + ui_home: cfg.server.ui.home().to_string(), } } @@ -214,6 +231,12 @@ fn diff( if previous.refresh_max_concurrent != next.refresh_max_concurrent { applied.push("server.refresh_max_concurrent".to_string()); } + if previous.ldap != next.ldap { + applied.push("server.ui.ldap".to_string()); + } + if previous.ui_home != next.ui_home { + applied.push("server.ui.home".to_string()); + } applied.sort(); report.applied = applied; diff --git a/src/application/credentials.rs b/src/application/credentials.rs index bfb3e38..cb733a7 100644 --- a/src/application/credentials.rs +++ b/src/application/credentials.rs @@ -1,5 +1,8 @@ use std::collections::HashMap; +use crate::domain::ldap::LdapConfig; +use crate::domain::project::GitProject; +use crate::domain::source::Source; use crate::ports::secrets::{SecretsError, SecretsPort}; // Resolves a list of credential_ids against the SecretsPort and combines @@ -32,6 +35,53 @@ pub async fn resolve_credentials( Ok(all_credentials) } +// What names a credential in its configuration: the sources that pass it +// to their connector, the projects that clone with it, and the directory +// login's service account. Sorted, so a page and a diff of it are stable. +pub struct CredentialUsers { + pub sources: Vec, + pub projects: Vec, + pub ldap: bool, +} + +pub fn users_of( + credential_id: &str, + sources: &HashMap, + projects: &HashMap, + ldap: Option<&LdapConfig>, +) -> CredentialUsers { + let mut source_ids: Vec = sources + .iter() + .filter(|(_, source)| source.credential_ids.iter().any(|c| c == credential_id)) + .map(|(id, _)| id.clone()) + .collect(); + source_ids.sort(); + let mut project_ids: Vec = projects + .iter() + .filter(|(_, project)| project.credential_id.as_deref() == Some(credential_id)) + .map(|(id, _)| id.clone()) + .collect(); + project_ids.sort(); + CredentialUsers { + sources: source_ids, + projects: project_ids, + ldap: ldap.is_some_and(|l| l.bind_credential == credential_id), + } +} + +// Whether a credential's secrets resolve right now, asked exactly as a sync +// asks (through the same resolver chain and its cache), so "resolves" here +// means a sync will get them. None when it does; otherwise the resolver's +// own message, which names what is missing (the env var, the file, the +// Vault path) and never a value. The values themselves are dropped here. +pub async fn check(secrets: &dyn SecretsPort, credential_id: &str) -> Option { + secrets + .resolve(credential_id) + .await + .err() + .map(|e| e.message) +} + #[cfg(test)] mod tests { use super::*; diff --git a/src/application/host_search.rs b/src/application/host_search.rs new file mode 100644 index 0000000..daf24cd --- /dev/null +++ b/src/application/host_search.rs @@ -0,0 +1,430 @@ +use std::collections::{BTreeSet, HashMap}; + +use crate::application::views::snapshot; +use crate::domain::endpoint::OutputEndpoint; +use crate::domain::source::{AdvertisedScopeRegistry, Source}; +use crate::domain::view::View; +use crate::ports::cache::CachePort; + +// "Where is this host?" across every source at once: which sources hold it, +// how old each one's data is, and which endpoints it reaches. The question an +// operator brings to the UI most, answered from the cache (no gather) so it +// is as cheap as a few map scans. + +// A query shorter than this matches too much to be useful (every host has an +// "a" in it), and costs a scan of everything to say so. +pub const MIN_QUERY: usize = 2; + +pub struct HostSource { + pub source_id: String, + pub age_seconds: u64, +} + +pub struct HostMatch { + pub host: String, + pub sources: Vec, + pub endpoints: Vec, +} + +pub struct SearchResult { + pub matches: Vec, + // How many hosts matched in all, before the limit: "20 of 312". + pub total: usize, +} + +// How well a host name matches: the whole name, its start, or anywhere. +// Lower sorts first, so typing a full name puts that host on top however +// many others contain it. +fn rank(name: &str, needle: &str) -> u8 { + if name == needle { + 0 + } else if name.starts_with(needle) { + 1 + } else { + 2 + } +} + +// Case-insensitive, on any part of the name. Only sources the caller may +// read are searched, and only endpoints they may run are named: search must +// not reveal a host's existence through a source the caller cannot open. +// What a search reads: the configured sources, views and endpoints, the +// cache, and the advertised scopes views resolve their members with. +pub struct Catalog<'a> { + pub sources: &'a HashMap, + pub views: &'a HashMap, + pub endpoints: &'a HashMap, + pub cache: &'a dyn CachePort, + pub scopes: &'a AdvertisedScopeRegistry, +} + +pub fn search( + query: &str, + limit: usize, + catalog: Catalog<'_>, + may_read: impl Fn(&str) -> bool, + may_run: impl Fn(&str) -> bool, +) -> SearchResult { + let Catalog { + sources, + views, + endpoints, + cache, + scopes, + } = catalog; + let needle = query.trim().to_lowercase(); + if needle.chars().count() < MIN_QUERY { + return SearchResult { + matches: Vec::new(), + total: 0, + }; + } + + // Each readable source's matching hosts, by exact name: a host is the + // same host wherever it is listed under the same name. + let mut found: HashMap> = HashMap::new(); + // The matching hosts of EVERY source, readable or not, for the endpoint + // check below: a key may run an endpoint whose limiting source it cannot + // open, and the endpoint's output would show the host all the same. + let mut held: HashMap<&str, BTreeSet> = HashMap::new(); + let mut ids: Vec<&String> = sources.keys().collect(); + ids.sort(); + for id in ids { + let Some(entry) = cache.get(id) else { continue }; + let readable = may_read(id); + let age_seconds = entry.age_seconds(); + let mut here = BTreeSet::new(); + for host in entry.dataset.hostvars.keys() { + if host.to_lowercase().contains(&needle) { + if readable { + found.entry(host.clone()).or_default().push(HostSource { + source_id: id.clone(), + age_seconds, + }); + } + here.insert(host.clone()); + } + } + held.insert(id.as_str(), here); + } + + // A view the caller may read, by the hosts IT serves: a key granted only + // a view finds its hosts here, listed under the view, and only those the + // view serves (its members' other hosts stay out of reach, as on the + // view's own routes). + let mut view_ids: Vec<&String> = views.keys().filter(|id| may_read(id)).collect(); + view_ids.sort(); + for id in view_ids { + let snap = snapshot(cache, sources, scopes, id, &views[id.as_str()]); + let age_seconds = snap.age_seconds(); + for host in snap.hosts() { + if host.to_lowercase().contains(&needle) { + found.entry(host.to_string()).or_default().push(HostSource { + source_id: id.clone(), + age_seconds, + }); + } + } + } + + let total = found.len(); + let mut hosts: Vec<(String, Vec)> = found.into_iter().collect(); + // Best match first; among names equal but for case, the one spelled as + // typed; then by name, so the order never depends on the map's. + let typed = query.trim(); + hosts.sort_by(|(a, _), (b, _)| { + let (la, lb) = (a.to_lowercase(), b.to_lowercase()); + rank(&la, &needle) + .cmp(&rank(&lb, &needle)) + .then_with(|| (a.as_str() != typed).cmp(&(b.as_str() != typed))) + .then_with(|| la.cmp(&lb)) + .then_with(|| a.cmp(b)) + }); + hosts.truncate(limit); + + let holds = |source_id: &str, host: &str| { + held.get(source_id) + .is_some_and(|hosts| hosts.contains(host)) + }; + let mut endpoint_ids: Vec<&String> = endpoints.keys().filter(|id| may_run(id)).collect(); + endpoint_ids.sort(); + + // An endpoint missing data from any source answers 503, not an + // inventory: it reaches no host, so it is left out entirely. + endpoint_ids.retain(|id| { + endpoints[id.as_str()] + .source_ids + .iter() + .all(|s| cache.get(s).is_some()) + }); + + // A builtin with settings of its own (a `group`, a host filter) keeps + // only some of its sources' hosts. Those are worked out the way a call + // works them out, once per search and only for such endpoints: an + // endpoint without settings is answered from the sources alone, cheaply. + let filtered: HashMap<&str, BTreeSet> = endpoint_ids + .iter() + .filter_map(|id| { + let endpoint = &endpoints[id.as_str()]; + if endpoint.output.is_none() || endpoint.config.is_empty() { + return None; + } + let datasets: HashMap> = + endpoint + .source_ids + .iter() + .filter_map(|s| cache.get(s).map(|e| (s.clone(), e.dataset.clone()))) + .collect(); + let datasets = match &endpoint.limit { + Some(limit) => crate::application::output::apply_limit(&datasets, limit), + None => datasets, + }; + let kept = crate::application::output::default_hosts(&datasets, &endpoint.config); + Some((id.as_str(), kept.into_iter().collect())) + }) + .collect(); + + let matches = hosts + .into_iter() + .map(|(host, sources)| { + // In an endpoint's output when one of its sources holds the host; + // under a limit, only when the limiting source does, since that + // source decides which hosts the output keeps at all; under its + // own settings, only when they keep it. + let endpoints = endpoint_ids + .iter() + .filter(|id| { + if let Some(kept) = filtered.get(id.as_str()) { + return kept.contains(&host); + } + let endpoint = &endpoints[id.as_str()]; + match endpoint + .limit + .as_ref() + .and_then(|l| l.by_hosts_from_inventory.as_deref()) + { + Some(limiting) => holds(limiting, &host), + None => endpoint.source_ids.iter().any(|s| holds(s, &host)), + } + }) + .map(|id| (*id).clone()) + .collect(); + HostMatch { + host, + sources, + endpoints, + } + }) + .collect(); + + SearchResult { matches, total } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::adapters::out::cache::memory::MemoryCache; + use crate::domain::cache_entry::CacheEntry; + use crate::domain::dataset::Dataset; + + fn source() -> Source { + serde_saphyr::from_str("name: x\nproject_id: p\nscript_path: s.py\nttl_seconds: 60\n") + .unwrap() + } + + fn endpoint(sources: &[&str], limit: Option<&str>) -> OutputEndpoint { + let mut yaml = format!( + "name: e\nsource_ids: [{}]\noutput: ansible\n", + sources.join(", ") + ); + if let Some(l) = limit { + yaml.push_str(&format!("limit:\n by_hosts_from_inventory: {l}\n")); + } + serde_saphyr::from_str(&yaml).unwrap() + } + + fn cache_with(entries: &[(&str, &[&str])]) -> MemoryCache { + let cache = MemoryCache::new(); + for (id, hosts) in entries { + let hostvars: serde_json::Map = hosts + .iter() + .map(|h| (h.to_string(), serde_json::json!({}))) + .collect(); + let dataset: Dataset = + serde_json::from_value(serde_json::json!({ "hostvars": hostvars, "groups": {} })) + .unwrap(); + cache.set(id, CacheEntry::new(dataset, 60)); + } + cache + } + + #[test] + fn finds_a_host_in_every_source_and_ranks_the_exact_name_first() { + let empty_views: HashMap = HashMap::new(); + let scopes = AdvertisedScopeRegistry::new(); + let sources: HashMap = [("src-a", source()), ("src-b", source())] + .map(|(k, v)| (k.to_string(), v)) + .into(); + let cache = cache_with(&[ + ("src-a", &["web1.example", "web10.example", "db1.example"]), + ("src-b", &["WEB1.example", "web1.example"]), + ]); + let found = search( + "web1.example", + 10, + Catalog { + sources: &sources, + views: &empty_views, + endpoints: &HashMap::new(), + cache: &cache, + scopes: &scopes, + }, + |_| true, + |_| true, + ); + assert_eq!(found.matches[0].host, "web1.example"); + let held: Vec<&str> = found.matches[0] + .sources + .iter() + .map(|s| s.source_id.as_str()) + .collect(); + assert_eq!(held, ["src-a", "src-b"]); + // Case-insensitive: WEB1 matches too, as its own name. + assert!(found.matches.iter().any(|m| m.host == "WEB1.example")); + assert!(!found.matches.iter().any(|m| m.host == "db1.example")); + } + + #[test] + fn a_source_the_key_may_not_read_is_not_searched() { + let empty_views: HashMap = HashMap::new(); + let scopes = AdvertisedScopeRegistry::new(); + let sources: HashMap = [("src-open", source()), ("src-closed", source())] + .map(|(k, v)| (k.to_string(), v)) + .into(); + let cache = cache_with(&[("src-open", &["a1"]), ("src-closed", &["a2"])]); + let found = search( + "a", + 10, + Catalog { + sources: &sources, + views: &empty_views, + endpoints: &HashMap::new(), + cache: &cache, + scopes: &scopes, + }, + |id| id == "src-open", + |_| true, + ); + assert_eq!(found.total, 0, "one character is below the minimum"); + let found = search( + "a2", + 10, + Catalog { + sources: &sources, + views: &empty_views, + endpoints: &HashMap::new(), + cache: &cache, + scopes: &scopes, + }, + |id| id == "src-open", + |_| true, + ); + assert_eq!( + found.total, 0, + "a host only in an unreadable source stays hidden" + ); + } + + #[test] + fn an_endpoint_under_a_limit_includes_only_the_limiting_sources_hosts() { + let empty_views: HashMap = HashMap::new(); + let scopes = AdvertisedScopeRegistry::new(); + let sources: HashMap = [("src-inv", source()), ("src-facts", source())] + .map(|(k, v)| (k.to_string(), v)) + .into(); + let endpoints: HashMap = [ + ( + "ep-limited", + endpoint(&["src-inv", "src-facts"], Some("src-inv")), + ), + ("ep-all", endpoint(&["src-inv", "src-facts"], None)), + ] + .map(|(k, v)| (k.to_string(), v)) + .into(); + let cache = cache_with(&[ + ("src-inv", &["in-both", "inv-only"]), + ("src-facts", &["in-both", "facts-only"]), + ]); + let reach = |q: &str| { + search( + q, + 10, + Catalog { + sources: &sources, + views: &empty_views, + endpoints: &endpoints, + cache: &cache, + scopes: &scopes, + }, + |_| true, + |_| true, + ) + .matches[0] + .endpoints + .clone() + }; + assert_eq!(reach("in-both"), ["ep-all", "ep-limited"]); + // Facts alone cannot add a host to a limited endpoint. + assert_eq!(reach("facts-only"), ["ep-all"]); + } + + // An endpoint's own settings decide too: a `group` it keeps leaves out a + // host its sources hold; and an endpoint missing a source answers 503, + // so it reaches nothing. + #[test] + fn an_endpoint_lists_a_host_only_if_its_own_output_would() { + let empty_views: HashMap = HashMap::new(); + let scopes = AdvertisedScopeRegistry::new(); + let sources: HashMap = [("src-inv", source()), ("src-gone", source())] + .map(|(k, v)| (k.to_string(), v)) + .into(); + let mut linux = endpoint(&["src-inv"], None); + linux + .config + .insert("group".to_string(), "linux".to_string()); + let endpoints: HashMap = [ + ("ep-linux", linux), + ("ep-broken", endpoint(&["src-inv", "src-gone"], None)), + ] + .map(|(k, v)| (k.to_string(), v)) + .into(); + let cache = MemoryCache::new(); + let dataset: Dataset = serde_json::from_value(serde_json::json!({ + "hostvars": { "web1": {}, "win1": {} }, + "groups": { "linux": { "hosts": ["web1"] } } + })) + .unwrap(); + cache.set("src-inv", CacheEntry::new(dataset, 60)); + + let reach = |q: &str| { + search( + q, + 10, + Catalog { + sources: &sources, + views: &empty_views, + endpoints: &endpoints, + cache: &cache, + scopes: &scopes, + }, + |_| true, + |_| true, + ) + .matches[0] + .endpoints + .clone() + }; + assert_eq!(reach("web1"), ["ep-linux"]); + assert!(reach("win1").is_empty(), "outside the endpoint's group"); + } +} diff --git a/src/application/login.rs b/src/application/login.rs new file mode 100644 index 0000000..7b0828a --- /dev/null +++ b/src/application/login.rs @@ -0,0 +1,364 @@ +use std::collections::HashMap; +use std::sync::Mutex; +use std::time::{Duration, Instant}; + +use crate::domain::ldap::{Grant, LdapConfig, grant_for}; +use crate::ports::directory::{DirectoryError, DirectoryLogin, DirectoryPort}; +use crate::ports::secrets::SecretsPort; + +// The use case "log a person in with their directory account". +// +// Three steps, each owned elsewhere: the service account comes from the +// secrets chain (like any connector credential), the directory checks the +// password, and the configured rules decide what the person may touch. This +// is where they meet, so the HTTP handler only translates a form. + +#[derive(Debug)] +pub struct LoggedIn { + pub username: String, + pub dn: String, + pub grant: Grant, +} + +#[derive(Debug, PartialEq)] +pub enum LoginError { + // Wrong password or unknown user — deliberately one answer. + InvalidCredentials, + // The password was right, but no `access` rule names this person. + NotAuthorized, + // The directory or the service account's secret could not be reached; + // the message is for the log. + Unavailable(String), + // This account failed too many times lately: refused without asking + // the directory, until the window passes. + TooManyAttempts, + // Too many logins already in flight. + Busy, +} + +// Failed passwords allowed per account in a window before the form stops +// asking the directory about it. Well under any sensible directory lockout, +// so a script cannot use this form to lock a real account out. +const MAX_FAILURES: u32 = 5; +const FAILURE_WINDOW: Duration = Duration::from_secs(15 * 60); +// Directory logins in flight at once: each is a TLS connection and two binds. +const MAX_CONCURRENT: usize = 4; +// Every failed login answers after this long in total, however far it got: +// an unknown user (one search) and a wrong password (one more bind) must +// take the same time, or the time says which usernames exist. +pub const FAILURE_DURATION: Duration = Duration::from_millis(1500); + +// What the login form guards itself with: recent failures per account, and +// a cap on concurrent directory logins. In memory and per process; it +// forgets on restart, which is acceptable for a rate limit. +pub struct LoginThrottle { + failures: Mutex>, + in_flight: tokio::sync::Semaphore, +} + +impl Default for LoginThrottle { + fn default() -> Self { + LoginThrottle { + failures: Mutex::new(HashMap::new()), + in_flight: tokio::sync::Semaphore::new(MAX_CONCURRENT), + } + } +} + +impl LoginThrottle { + // Accounts are named however they were typed; case is not identity. + fn key(username: &str) -> String { + username.trim().to_lowercase() + } + + // Whether this account has failed too often lately to be asked about. + fn blocked(&self, username: &str) -> bool { + let mut failures = self.failures.lock().expect("login throttle lock"); + match failures.get(&Self::key(username)) { + Some((count, since)) if since.elapsed() < FAILURE_WINDOW => *count >= MAX_FAILURES, + Some(_) => { + failures.remove(&Self::key(username)); + false + } + None => false, + } + } + + fn failed(&self, username: &str) { + let mut failures = self.failures.lock().expect("login throttle lock"); + // Forget windows that have closed, so the map does not grow with + // every name ever typed at the form. + failures.retain(|_, (_, since)| since.elapsed() < FAILURE_WINDOW); + let entry = failures + .entry(Self::key(username)) + .or_insert((0, Instant::now())); + entry.0 += 1; + } + + fn succeeded(&self, username: &str) { + self.failures + .lock() + .expect("login throttle lock") + .remove(&Self::key(username)); + } +} + +// ldap_login behind the throttle: refuses an account that has failed too +// often without asking the directory, caps concurrent logins, and makes +// every wrong password take the same total time. +pub async fn throttled_ldap_login( + throttle: &LoginThrottle, + secrets: &dyn SecretsPort, + directory: &dyn DirectoryPort, + config: &LdapConfig, + username: &str, + password: &str, +) -> Result { + let started = Instant::now(); + if throttle.blocked(username) { + return Err(LoginError::TooManyAttempts); + } + let Ok(_permit) = + tokio::time::timeout(Duration::from_secs(5), throttle.in_flight.acquire()).await + else { + return Err(LoginError::Busy); + }; + + let result = ldap_login(secrets, directory, config, username, password).await; + match &result { + Err(LoginError::InvalidCredentials) => { + throttle.failed(username); + tokio::time::sleep(FAILURE_DURATION.saturating_sub(started.elapsed())).await; + } + Ok(_) | Err(LoginError::NotAuthorized) => throttle.succeeded(username), + _ => {} + } + result +} + +pub async fn ldap_login( + secrets: &dyn SecretsPort, + directory: &dyn DirectoryPort, + config: &LdapConfig, + username: &str, + password: &str, +) -> Result { + let username = username.trim(); + + let bind = secrets + .resolve(&config.bind_credential) + .await + .map_err(|e| { + LoginError::Unavailable(format!( + "LDAP bind credential '{}': {}", + config.bind_credential, e.message + )) + })?; + let (Some(bind_dn), Some(bind_password)) = (bind.get("username"), bind.get("password")) else { + return Err(LoginError::Unavailable(format!( + "LDAP bind credential '{}' must provide `username` (the bind DN) and `password`", + config.bind_credential + ))); + }; + + let user = directory + .authenticate(DirectoryLogin { + config, + bind_dn, + bind_password, + username, + password, + }) + .await + .map_err(|e| match e { + DirectoryError::InvalidCredentials => LoginError::InvalidCredentials, + DirectoryError::Unavailable(message) => LoginError::Unavailable(message), + })?; + + // The directory's name for the account, not the typed spelling: rules + // and the session name follow the account (see `username_attribute`). + let username = user.username.as_deref().unwrap_or(username); + + let grant = grant_for(&config.access, username, &user.member_of).ok_or_else(|| { + tracing::warn!( + username, + groups = user.member_of.len(), + "LDAP login refused: no access rule names this user or their groups" + ); + LoginError::NotAuthorized + })?; + + Ok(LoggedIn { + username: username.to_string(), + dn: user.dn, + grant, + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::adapters::out::secrets::mock::MockSecrets; + use crate::domain::api_key::ApiKeyRole; + use crate::domain::ldap::AccessRule; + use crate::ports::directory::{DirectoryFuture, DirectoryUser}; + + // A directory that knows one person: jdoe / right-password, in DL-ops. + struct OnePerson; + + impl DirectoryPort for OnePerson { + fn authenticate<'a>(&'a self, request: DirectoryLogin<'a>) -> DirectoryFuture<'a> { + Box::pin(async move { + assert_eq!(request.bind_dn, "CN=svc-unified"); + if request.username == "jdoe" && request.password == "right-password" { + Ok(DirectoryUser { + dn: "CN=John Doe,OU=Users,DC=x".to_string(), + username: Some("jdoe".to_string()), + member_of: vec!["CN=DL-ops,OU=Groups,DC=x".to_string()], + }) + } else { + Err(DirectoryError::InvalidCredentials) + } + }) + } + } + + fn config(groups: &[&str]) -> LdapConfig { + LdapConfig { + url: "ldaps://ldap.example.com".to_string(), + starttls: false, + ca_file: None, + bind_credential: "cred-ldap".to_string(), + search_base: "DC=x".to_string(), + user_filter: "(sAMAccountName={username})".to_string(), + group_attribute: "memberOf".to_string(), + username_attribute: "sAMAccountName".to_string(), + timeout_seconds: 5, + access: vec![AccessRule { + name: "ops".to_string(), + groups: groups.iter().map(|g| g.to_string()).collect(), + users: Vec::new(), + role: ApiKeyRole::Admin, + sources: Vec::new(), + endpoints: Vec::new(), + }], + } + } + + fn secrets() -> MockSecrets { + MockSecrets::new().with( + "cred-ldap", + [("username", "CN=svc-unified"), ("password", "svc-secret")], + ) + } + + #[tokio::test] + async fn a_member_of_an_admin_group_logs_in_as_admin() { + let logged = ldap_login( + &secrets(), + &OnePerson, + &config(&["DL-ops"]), + " jdoe ", + "right-password", + ) + .await + .expect("login"); + assert_eq!(logged.username, "jdoe"); + assert_eq!(logged.grant, Grant::Admin); + } + + #[tokio::test] + async fn a_wrong_password_is_invalid_credentials() { + let err = ldap_login(&secrets(), &OnePerson, &config(&["DL-ops"]), "jdoe", "nope") + .await + .unwrap_err(); + assert_eq!(err, LoginError::InvalidCredentials); + } + + // The directory accepting the password is not access: only a rule is. + #[tokio::test] + async fn a_right_password_without_a_rule_is_not_authorized() { + let err = ldap_login( + &secrets(), + &OnePerson, + &config(&["DL-other"]), + "jdoe", + "right-password", + ) + .await + .unwrap_err(); + assert_eq!(err, LoginError::NotAuthorized); + } + + #[tokio::test] + async fn a_missing_bind_credential_is_unavailable_and_named() { + let err = ldap_login( + &MockSecrets::new(), + &OnePerson, + &config(&["DL-ops"]), + "jdoe", + "x", + ) + .await + .unwrap_err(); + let LoginError::Unavailable(message) = err else { + panic!("expected Unavailable, got {err:?}"); + }; + assert!(message.contains("cred-ldap"), "{message}"); + } + + // The directory names whoever the filter found; rules follow that name. + struct Named(&'static str); + + impl DirectoryPort for Named { + fn authenticate<'a>(&'a self, _request: DirectoryLogin<'a>) -> DirectoryFuture<'a> { + let name = self.0; + Box::pin(async move { + Ok(DirectoryUser { + dn: format!("CN={name},DC=x"), + username: Some(name.to_string()), + member_of: Vec::new(), + }) + }) + } + } + + fn named_rule(users: &[&str]) -> LdapConfig { + let mut cfg = config(&[]); + cfg.access[0].groups.clear(); + cfg.access[0].users = users.iter().map(|u| u.to_string()).collect(); + cfg + } + + #[tokio::test] + async fn user_rules_match_the_account_the_directory_found_not_the_typing() { + let secrets = MockSecrets::new().with( + "cred-ldap", + [("username", "CN=svc-unified"), ("password", "svc")], + ); + // Typed someone else's name, but the directory says it is "mallory": + // the rule written for "jdoe" must not apply. + let refused = ldap_login( + &secrets, + &Named("mallory"), + &named_rule(&["jdoe"]), + "jdoe", + "pw", + ) + .await; + assert_eq!(refused.unwrap_err(), LoginError::NotAuthorized); + + // Typed an e-mail address the filter maps to jdoe: jdoe's rule applies, + // and the session is named after the account. + let logged = ldap_login( + &secrets, + &Named("jdoe"), + &named_rule(&["jdoe"]), + "j.doe@x.com", + "pw", + ) + .await + .expect("jdoe's rule"); + assert_eq!(logged.username, "jdoe"); + } +} diff --git a/src/application/mod.rs b/src/application/mod.rs index 3a82c1f..3a7d5ab 100644 --- a/src/application/mod.rs +++ b/src/application/mod.rs @@ -7,9 +7,12 @@ pub mod config; pub mod credentials; pub mod enrich; +pub mod host_search; +pub mod login; pub mod output; pub mod projects; pub mod refresh; pub mod scripts; pub mod sync; +pub mod topology; pub mod views; diff --git a/src/application/output.rs b/src/application/output.rs index 69964d0..568ed7e 100644 --- a/src/application/output.rs +++ b/src/application/output.rs @@ -282,6 +282,30 @@ fn merge_and_filter( (hostvars, groups) } +// How many hosts and groups a call with no parameters of its own returns: +// the same merge and filter as the builtins, without rendering anything. For +// a page that wants the endpoint's size without fetching its output. +pub fn counts( + datasets: &HashMap>, + config: &HashMap, +) -> (usize, usize) { + let (hostvars, groups) = merge_and_filter(datasets, config, &serde_json::json!({})); + (hostvars.len(), groups.len()) +} + +// The hosts a call with no parameters of its own returns, by name: the same +// merge and filter, kept to the names. For a question about one host ("does +// this endpoint include it?") without rendering anything. +pub fn default_hosts( + datasets: &HashMap>, + config: &HashMap, +) -> HashSet { + merge_and_filter(datasets, config, &serde_json::json!({})) + .0 + .into_keys() + .collect() +} + // Render the merged datasets as Ansible dynamic inventory JSON // (`_meta.hostvars` plus one key per group). Filters: see merge_and_filter. pub fn render_ansible( diff --git a/src/application/topology.rs b/src/application/topology.rs new file mode 100644 index 0000000..fbe511a --- /dev/null +++ b/src/application/topology.rs @@ -0,0 +1,276 @@ +use std::collections::HashMap; +use std::sync::Arc; + +use crate::domain::source::Source; +use crate::domain::sync_health::SyncHealthRegistry; +use crate::domain::topology::{Peer, PeerSource, Topology, peers_of}; +use crate::ports::cache::CachePort; +use crate::ports::peer::{PeerPort, PeerRequest}; +use crate::ports::secrets::SecretsPort; + +// The use case "describe the federation from here". +// +// Local facts first (which remote sources read from which URL, and how their +// pulls are going), then each peer asked to describe itself, concurrently, +// with one hop less to spend. A peer that cannot be asked is shown with the +// reason; the map never fails because one instance is down — that is exactly +// when someone opens it. + +// Most an instance will pass on: a central and its edges is two levels, and +// a loop of instances that federate each other must end somewhere. +pub const MAX_DEPTH: u8 = 3; + +// A topology request waits on its slowest peer; a source's own sync timeout +// (often minutes, for big datasets) would hold the page that long. +const PEER_TIMEOUT_SECONDS: u64 = 8; + +// How long to wait for a peer that will itself ask `depth` more levels: its +// own answer, plus the time it may spend waiting on each level below. A flat +// timeout let one dead instance two hops away hide a healthy one: the middle +// peer was still waiting on it when our wait for the middle peer ran out. +fn peer_timeout(depth: u8) -> u64 { + PEER_TIMEOUT_SECONDS * (u64::from(depth) + 1) +} + +pub struct Inputs<'a> { + pub sources: &'a HashMap, + pub cache: &'a dyn CachePort, + pub health: &'a SyncHealthRegistry, + pub secrets: &'a dyn SecretsPort, + pub peers: Arc, +} + +pub async fn describe(inputs: Inputs<'_>, visible: impl Fn(&str) -> bool, depth: u8) -> Topology { + let depth = depth.min(MAX_DEPTH); + let visible_count = inputs.sources.keys().filter(|id| visible(id)).count(); + let grouped = peers_of(inputs.sources, &visible); + + let mut peers: Vec = Vec::with_capacity(grouped.len()); + let mut asks = tokio::task::JoinSet::new(); + let mut to_ask = Vec::new(); + let mut asked = Vec::new(); + + for (index, (url, members)) in grouped.into_iter().enumerate() { + let sources = members + .iter() + .map(|(id, source)| { + let entry = inputs.cache.get(id); + let health = inputs.health.get(id).unwrap_or_default(); + PeerSource { + source_id: id.to_string(), + remote_source: source.script_path.clone(), + cached: entry.is_some(), + hosts: entry.as_ref().map(|e| e.dataset.hostvars.len()), + groups: entry.as_ref().map(|e| e.dataset.groups.len()), + age_seconds: entry.as_ref().map(|e| e.age_seconds()), + consecutive_failures: health.consecutive_failures, + last_error: health.last_error, + } + }) + .collect(); + + if depth == 0 { + peers.push(Peer { + url, + sources, + instance: None, + error: Some("not asked: depth limit reached".to_string()), + }); + continue; + } + + // The key and TLS setting the sources reading this peer already use: + // whatever lets them pull data lets the map ask who it is. Resolved + // below, once every peer is listed. + let (_, first) = members[0]; + to_ask.push((index, url.clone(), first)); + peers.push(Peer { + url, + sources, + instance: None, + error: None, + }); + } + + // One after another: resolutions come from the secrets cache (env vars + // are instant), and it is the peer requests below that run concurrently. + let mut keys = Vec::with_capacity(to_ask.len()); + for (_, _, first) in &to_ask { + keys.push( + crate::application::credentials::resolve_credentials( + inputs.secrets, + &first.credential_ids, + ) + .await, + ); + } + + for ((index, url, first), resolved) in to_ask.into_iter().zip(keys) { + let api_key = match resolved { + Ok(credentials) => credentials + .get("token") + .or_else(|| credentials.get("TOKEN")) + .or_else(|| credentials.get("api_key")) + .cloned(), + Err(e) => { + peers[index].error = Some(format!("not asked: {}", e.message)); + continue; + } + }; + let request = PeerRequest { + url, + api_key, + insecure_tls: first + .config + .get("insecure_tls") + .is_some_and(|v| v == "true"), + timeout_seconds: peer_timeout(depth - 1), + depth: depth - 1, + }; + let port = Arc::clone(&inputs.peers); + asks.spawn(async move { (index, port.describe(request).await) }); + asked.push(index); + } + + // Every grouped peer was pushed in order, asked or not, so the index a + // task carries is its place in `peers` whatever order answers arrive in. + while let Some(joined) = asks.join_next().await { + let Ok((index, answer)) = joined else { + continue; + }; + if let Some(peer) = peers.get_mut(index) { + match answer { + Ok(topology) => peer.instance = Some(Box::new(topology)), + Err(reason) => peer.error = Some(reason), + } + } + } + + // A task that panicked returned no index: its peer would be left with + // neither an answer nor a reason, which the contract never allows. + for index in asked { + let peer = &mut peers[index]; + if peer.instance.is_none() && peer.error.is_none() { + peer.error = Some("not described: an internal error stopped the request".to_string()); + } + } + + Topology { + version: env!("CARGO_PKG_VERSION").to_string(), + sources: Some(visible_count), + peers, + } +} + +#[cfg(test)] +mod tests { + use super::*; + + // A peer that will ask its own peers gets their wait on top of its own, + // or a dead instance two hops away hides the healthy one in between. + #[test] + fn a_peer_asking_further_gets_longer_than_its_own_peers() { + assert_eq!(peer_timeout(0), PEER_TIMEOUT_SECONDS); + assert!(peer_timeout(1) > peer_timeout(0) + PEER_TIMEOUT_SECONDS - 1); + } + use crate::adapters::out::cache::memory::MemoryCache; + use crate::adapters::out::secrets::mock::MockSecrets; + use crate::ports::peer::PeerFuture; + + // dc1 answers as an edge with no peers of its own; dc2 is down. + struct Peers; + + impl PeerPort for Peers { + fn describe(&self, request: PeerRequest) -> PeerFuture { + Box::pin(async move { + assert_eq!(request.api_key.as_deref(), Some("edge-key")); + if request.url.contains("dc2") { + return Err("unreachable".to_string()); + } + Ok(Topology { + version: "0.33.0".to_string(), + sources: Some(3), + peers: Vec::new(), + }) + }) + } + } + + fn remote(url: &str) -> Source { + serde_saphyr::from_str(&format!( + "name: x\nproject_id: p\nscript_path: src-ssh-facts\nttl_seconds: 60\nconnector_type: remote\n\ + credential_ids: [cred-edge]\nconfig:\n url: \"{url}\"\n" + )) + .unwrap() + } + + fn sources() -> HashMap { + [ + ("src-edge-dc1", remote("https://dc1.example:8443")), + ("src-edge-dc2", remote("https://dc2.example:8443")), + ] + .into_iter() + .map(|(id, s)| (id.to_string(), s)) + .collect() + } + + #[tokio::test] + async fn every_peer_is_described_or_says_why_not() { + let sources = sources(); + let cache = MemoryCache::new(); + let health = SyncHealthRegistry::new(); + let secrets = MockSecrets::new().with("cred-edge", [("token", "edge-key")]); + let topology = describe( + Inputs { + sources: &sources, + cache: &cache, + health: &health, + secrets: &secrets, + peers: Arc::new(Peers), + }, + |_| true, + 2, + ) + .await; + + assert_eq!(topology.sources, Some(2)); + assert_eq!(topology.peers.len(), 2); + let dc1 = &topology.peers[0]; + assert_eq!(dc1.url, "https://dc1.example:8443"); + assert_eq!(dc1.instance.as_ref().unwrap().version, "0.33.0"); + assert_eq!(dc1.sources[0].remote_source, "src-ssh-facts"); + let dc2 = &topology.peers[1]; + assert!(dc2.instance.is_none()); + assert_eq!(dc2.error.as_deref(), Some("unreachable")); + } + + // Depth 0 maps this instance's peers without asking them anything: the + // hop budget ran out, and a loop of instances must end. + #[tokio::test] + async fn no_depth_left_means_no_peer_is_asked() { + let sources = sources(); + let cache = MemoryCache::new(); + let health = SyncHealthRegistry::new(); + let secrets = MockSecrets::new(); + let topology = describe( + Inputs { + sources: &sources, + cache: &cache, + health: &health, + secrets: &secrets, + peers: Arc::new(Peers), + }, + |_| true, + 0, + ) + .await; + assert!(topology.peers.iter().all(|p| p.instance.is_none())); + assert!( + topology.peers[0] + .error + .as_deref() + .unwrap() + .contains("depth") + ); + } +} diff --git a/src/config.rs b/src/config.rs index 0eb8eee..b5a82f2 100644 --- a/src/config.rs +++ b/src/config.rs @@ -254,21 +254,80 @@ pub fn is_config_file(name: &str) -> bool { // keys is how a pipeline comes to believe it changed a port it did not // change — so they are diffed and NAMED in the reload report instead. // The browsable API's settings. -#[derive(Deserialize, Default, Clone, PartialEq, Debug)] +#[derive(Deserialize, Clone, PartialEq, Debug)] #[serde(deny_unknown_fields)] pub struct UiConfig { #[serde(default)] pub enabled: bool, - // How long a browser session lives. Read per login, so it reloads. + // How long a browser session lives. Fixed when the routes are built (the + // login handler receives it as an extension), so it is restart-only and + // listed in RestartOnlySettings: a reload that changes it says so. #[serde(default = "default_session_ttl_seconds")] pub session_ttl_seconds: u64, + + // Log in with a directory account instead of pasting an API key. Read per + // login from the configuration snapshot, so changing the server or the + // access rules is a reload — which also ends the LDAP sessions opened + // under the old rules (application::config::apply). + #[serde(default)] + pub ldap: Option, + + // Where a sign-in lands when nothing asked for a page: someone opening + // /api/v1/login directly, or signing in again after signing out. Unset, + // it is "/" when this instance serves the UI (`assets_dir`) and the API's + // own browsable pages otherwise. Read per login, so it reloads. + #[serde(default)] + pub home: Option, + + // A built unified-ui (the directory holding its index.html), served at + // "/": the UI and the API on one address, no proxy in front. The image + // ships one and sets this. Restart-only: the router is built once. + #[serde(default)] + pub assets_dir: Option, +} + +impl UiConfig { + pub fn home(&self) -> &str { + match (&self.home, &self.assets_dir) { + (Some(home), _) => home, + (None, Some(_)) => "/", + (None, None) => "/api/v1/", + } + } } fn default_session_ttl_seconds() -> u64 { 3600 } +// A path on THIS origin, and nothing a browser could read as another host: +// `//evil.example` is protocol-relative, a backslash is read as a slash, and +// browsers strip tabs and newlines before reading a URL, so "/\t/evil" would +// arrive as "//evil". Shared by the login's `next` and `server.ui.home`, so +// the redirect after a login can never be turned into an open redirect. +pub fn is_local_path(path: &str) -> bool { + path.starts_with('/') + && !path.starts_with("//") + && !path.contains('\\') + && !path.chars().any(|c| c.is_control() || c.is_whitespace()) +} + +// By hand rather than derived: a derived Default gives a session TTL of zero +// (serde's default only applies when parsing), so a UiConfig built in code +// issued sessions that were expired the moment they were created. +impl Default for UiConfig { + fn default() -> Self { + Self { + enabled: false, + session_ttl_seconds: default_session_ttl_seconds(), + ldap: None, + home: None, + assets_dir: None, + } + } +} + #[derive(Clone, PartialEq, Debug)] pub struct RestartOnlySettings { pub host: String, @@ -278,6 +337,8 @@ pub struct RestartOnlySettings { pub projects_dir: String, pub config_api_enabled: bool, pub ui_enabled: bool, + pub session_ttl_seconds: u64, + pub ui_assets_dir: Option, } impl RestartOnlySettings { @@ -294,6 +355,8 @@ impl RestartOnlySettings { projects_dir: cfg.projects_config.dir.clone(), config_api_enabled: cfg.config_api.enabled, ui_enabled: cfg.server.ui.enabled, + session_ttl_seconds: cfg.server.ui.session_ttl_seconds, + ui_assets_dir: cfg.server.ui.assets_dir.clone(), } } @@ -319,6 +382,14 @@ impl RestartOnlySettings { "config_api.enabled", ); check(self.ui_enabled != other.ui_enabled, "server.ui.enabled"); + check( + self.session_ttl_seconds != other.session_ttl_seconds, + "server.ui.session_ttl_seconds", + ); + check( + self.ui_assets_dir != other.ui_assets_dir, + "server.ui.assets_dir", + ); keys } } @@ -818,6 +889,115 @@ impl AppConfig { } } + if let Some(ldap) = &self.server.ui.ldap { + errors.extend(self.ldap_errors(ldap)); + } + + if let Some(home) = &self.server.ui.home + && !is_local_path(home) + { + errors.push(format!( + "server.ui.home '{home}' must be a path on this server, such as \"/\" — \ + a sign-in redirects there, and another host would make it an open redirect" + )); + } + + // The UI signs in through the API's own login page and its session + // cookie, which exist only with the browsable UI on. + if self.server.ui.assets_dir.is_some() && !self.server.ui.enabled { + errors.push( + "server.ui.assets_dir is set but server.ui.enabled is false — the UI \ + signs in through the login page, which only exists with the UI enabled" + .to_string(), + ); + } + + errors + } + + // Everything about `server.ui.ldap` that can be known without contacting + // the directory, so a bad push fails validation instead of every login. + fn ldap_errors(&self, ldap: &crate::domain::ldap::LdapConfig) -> Vec { + let mut errors = Vec::new(); + + // The login form and the session are the browsable API's; without it + // there is nothing for LDAP to log anybody into. + if !self.server.ui.enabled { + errors.push( + "server.ui.ldap is set but server.ui.enabled is false — LDAP login is \ + part of the browsable UI" + .to_string(), + ); + } + + let url = ldap.url.to_ascii_lowercase(); + if url.starts_with("ldap://") { + if !ldap.starttls { + errors.push(format!( + "server.ui.ldap.url '{}' is plain LDAP — every password would cross \ + the network in the clear; use ldaps:// or set starttls: true", + ldap.url + )); + } + } else if url.starts_with("ldaps://") { + if ldap.starttls { + errors.push( + "server.ui.ldap: starttls applies to ldap:// URLs; an ldaps:// \ + connection is already TLS" + .to_string(), + ); + } + } else { + errors.push(format!( + "server.ui.ldap.url '{}' must start with ldaps:// or ldap://", + ldap.url + )); + } + + if !self.credentials.contains_key(&ldap.bind_credential) { + errors.push(format!( + "server.ui.ldap.bind_credential references unknown credential '{}'", + ldap.bind_credential + )); + } + if !ldap.user_filter.contains("{username}") { + errors.push(format!( + "server.ui.ldap.user_filter '{}' has no {{username}} placeholder — it \ + would match the same entry whoever logs in", + ldap.user_filter + )); + } + if ldap.access.is_empty() { + errors.push( + "server.ui.ldap.access is empty — nobody could log in; name the groups \ + or users who may" + .to_string(), + ); + } + for rule in &ldap.access { + if rule.groups.is_empty() && rule.users.is_empty() { + errors.push(format!( + "server.ui.ldap.access '{}' names no groups and no users", + rule.name + )); + } + for id in &rule.sources { + if !self.sources.contains_key(id) && !self.views.contains_key(id) { + errors.push(format!( + "server.ui.ldap.access '{}' references unknown source '{}'", + rule.name, id + )); + } + } + for id in &rule.endpoints { + if !self.endpoints.contains_key(id) { + errors.push(format!( + "server.ui.ldap.access '{}' references unknown endpoint '{}'", + rule.name, id + )); + } + } + } errors } @@ -1415,6 +1595,70 @@ mod tests { assert!(err.contains("not an ssh source"), "{}", err); } + // The LDAP block is checked before anyone tries to log in: a plaintext + // URL, a service account that does not exist, or a UI that is off are + // all configuration errors, not login failures. + #[test] + fn validate_ldap_rules() { + let check = |ui: &str, credentials: &str| { + let dir = tempfile::tempdir().unwrap(); + fs::write( + dir.path().join("config.yaml"), + format!("server:\n host: \"127.0.0.1\"\n port: 9090\n ui:\n{ui}"), + ) + .unwrap(); + fs::write(dir.path().join("credentials.yaml"), credentials).unwrap(); + load_config(dir.path().to_str().unwrap()) + .err() + .map(|e| e.to_string()) + }; + let cred = + "cred-ldap:\n name: \"LDAP\"\n type: username_password\n env_prefix: \"LDAP\"\n"; + let ldap = |url: &str, extra: &str| { + format!( + " enabled: true\n ldap:\n url: \"{url}\"\n bind_credential: cred-ldap\n \ + search_base: \"DC=x\"\n access:\n - name: ops\n groups: [DL-ops]\n \ + role: admin\n{extra}" + ) + }; + + assert_eq!(check(&ldap("ldaps://ldap.example.com", ""), cred), None); + assert_eq!( + check( + &ldap("ldap://ldap.example.com", " starttls: true\n"), + cred + ), + None + ); + + let err = check(&ldap("ldap://ldap.example.com", ""), cred).expect("plaintext refused"); + assert!(err.contains("in the clear"), "{err}"); + + let err = check(&ldap("ldaps://ldap.example.com", ""), "").expect("unknown credential"); + assert!(err.contains("unknown credential 'cred-ldap'"), "{err}"); + + let off = ldap("ldaps://ldap.example.com", "").replace("enabled: true", "enabled: false"); + let err = check(&off, cred).expect("ui off"); + assert!(err.contains("server.ui.enabled is false"), "{err}"); + + // A UI to serve needs the login page it signs in through. + let err = check(" enabled: false\n assets_dir: \"/srv/ui\"\n", "") + .expect("assets without the UI refused"); + assert!(err.contains("server.ui.assets_dir"), "{err}"); + assert_eq!( + check(" enabled: true\n assets_dir: \"/srv/ui\"\n", ""), + None + ); + + // Where a login lands must stay on this server: it is a redirect. + assert_eq!(check(" enabled: true\n home: \"/\"\n", ""), None); + for home in ["https://evil.example", "//evil.example", "inventory"] { + let err = check(&format!(" enabled: true\n home: \"{home}\"\n"), "") + .expect("an off-site home refused"); + assert!(err.contains("server.ui.home"), "{err}"); + } + } + #[test] fn validate_catches_project_with_unknown_credential() { let dir = tempfile::tempdir().unwrap(); diff --git a/src/domain/api_key.rs b/src/domain/api_key.rs index 57b0e2e..d56101c 100644 --- a/src/domain/api_key.rs +++ b/src/domain/api_key.rs @@ -1,4 +1,4 @@ -use serde::Deserialize; +use serde::{Deserialize, Serialize}; // Configuration shape of one API key — api_keys.yaml. // @@ -35,7 +35,7 @@ pub struct ApiKeyDef { // `rename_all = "lowercase"` lets the YAML say `role: admin` instead of // `role: Admin` — config files shouldn't care about Rust naming style. -#[derive(Debug, Deserialize, Clone, Copy, PartialEq, Default)] +#[derive(Debug, Deserialize, Serialize, Clone, Copy, PartialEq, Default)] #[serde(rename_all = "lowercase")] pub enum ApiKeyRole { Admin, diff --git a/src/domain/credential.rs b/src/domain/credential.rs index 56db7de..609e707 100644 --- a/src/domain/credential.rs +++ b/src/domain/credential.rs @@ -47,3 +47,75 @@ pub struct Credential { #[serde(default)] pub file_keys: HashMap, } + +impl CredentialType { + // As written after `type:` in credentials.yaml. + pub fn name(&self) -> &'static str { + match self { + CredentialType::UsernamePassword => "username_password", + CredentialType::Token => "token", + CredentialType::SshKey => "ssh_key", + } + } +} + +impl Credential { + // Where the secret values are read from, in the order the resolver + // chain decides it: a Vault path wins over everything, then env vars, + // then a JSON file. "files" is a credential made only of file_keys + // (paths handed to the script, an SSH key, a certificate). + pub fn backend(&self) -> &'static str { + if self.vault_path.is_some() { + "vault" + } else if self.env_prefix.is_some() { + "env" + } else if self.secret_file.is_some() { + "file" + } else { + "files" + } + } + + // The environment variables an env-backed credential reads, sorted: + // PREFIX_FIELD for each secret key, spelled exactly as the env resolver + // spells them. Empty for any other backend. + pub fn env_vars(&self) -> Vec { + let (Some(prefix), None) = (&self.env_prefix, &self.vault_path) else { + return Vec::new(); + }; + let mut vars: Vec = self + .secret_keys + .values() + .map(|suffix| format!("{}_{}", prefix, suffix)) + .collect(); + vars.sort(); + vars + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn credential(yaml: &str) -> Credential { + serde_saphyr::from_str(yaml).unwrap() + } + + #[test] + fn env_vars_are_spelled_as_the_resolver_reads_them() { + let c = credential( + "name: x\ntype: username_password\nenv_prefix: D42\nsecret_keys:\n username: USER\n password: PASS\n", + ); + assert_eq!(c.backend(), "env"); + assert_eq!(c.env_vars(), vec!["D42_PASS", "D42_USER"]); + } + + #[test] + fn a_vault_path_wins_and_reads_no_env_vars() { + let c = credential( + "name: x\ntype: token\nvault_path: pcd/github\nenv_prefix: GH\nsecret_keys:\n token: TOKEN\n", + ); + assert_eq!(c.backend(), "vault"); + assert!(c.env_vars().is_empty()); + } +} diff --git a/src/domain/endpoint.rs b/src/domain/endpoint.rs index f03dd9d..a940a73 100644 --- a/src/domain/endpoint.rs +++ b/src/domain/endpoint.rs @@ -70,6 +70,18 @@ pub enum OutputFormat { Csv, } +impl OutputFormat { + // As written after `output:` in endpoints.yaml, for anything that shows + // the configuration back (the endpoints list). + pub fn name(self) -> &'static str { + match self { + OutputFormat::Ansible => "ansible", + OutputFormat::Json => "json", + OutputFormat::Csv => "csv", + } + } +} + // A constructed inventory: merge everything the endpoint's sources carry, then // hand back only part of it. // diff --git a/src/domain/ldap.rs b/src/domain/ldap.rs new file mode 100644 index 0000000..f8e59e6 --- /dev/null +++ b/src/domain/ldap.rs @@ -0,0 +1,284 @@ +use serde::{Deserialize, Serialize}; +use std::collections::HashSet; + +use crate::domain::api_key::ApiKeyRole; + +// LDAP login for the browsable API — config.yaml, `server.ui.ldap`. +// +// People log in with their directory account; what they may touch is decided +// HERE, by the `access` rules, in the same vocabulary api_keys.yaml uses +// (admin, or restricted to listed sources and endpoints). The directory says +// who someone is and which groups they belong to; it never decides access on +// its own, so a new group in AD grants nothing until a rule names it. +// +// The service account's password is not in this block: `bind_credential` +// names a credentials.yaml entry, resolved through the same secrets chain +// (env, file, Vault) every connector credential goes through. +#[derive(Debug, Deserialize, Serialize, Clone, PartialEq)] +#[serde(deny_unknown_fields)] +pub struct LdapConfig { + // ldaps://host:636, or ldap://host:389 with `starttls: true`. A plain + // ldap:// without StartTLS is refused at validation: it would send every + // password that logs in across the network in the clear. + pub url: String, + + #[serde(default)] + pub starttls: bool, + + // PEM file with the CA that signed the directory's certificate, for a + // private CA. Unset = the system's trust store. + #[serde(default)] + pub ca_file: Option, + + // credentials.yaml id whose `username` is the bind DN of the service + // account that looks users up, and whose `password` is its password. + pub bind_credential: String, + + // Where users are searched for, e.g. "OU=Users,DC=example,DC=com". + pub search_base: String, + + // `{username}` is replaced by what was typed, escaped (RFC 4515) so a + // username cannot rewrite the filter. Active Directory's default. + #[serde(default = "default_user_filter")] + pub user_filter: String, + + // Attribute listing the DNs of the user's groups. + #[serde(default = "default_group_attribute")] + pub group_attribute: String, + // Attribute holding the account's own name. Access rules' `users:` and + // the session's name use ITS value, not what was typed: with a filter + // that matches on several attributes (an e-mail address, say), what was + // typed is not the account's name, and must not be able to match a rule + // written for someone else's. + #[serde(default = "default_username_attribute")] + pub username_attribute: String, + + #[serde(default = "default_timeout_seconds")] + pub timeout_seconds: u64, + + // Who may log in, and with what. Evaluated top to bottom, all matches + // combined: admin anywhere is admin; otherwise the scoped lists are + // unioned. Matching nothing means the login is refused, even with a + // correct password. + #[serde(default)] + pub access: Vec, +} + +fn default_user_filter() -> String { + "(sAMAccountName={username})".to_string() +} + +fn default_username_attribute() -> String { + "sAMAccountName".to_string() +} + +fn default_group_attribute() -> String { + "memberOf".to_string() +} + +fn default_timeout_seconds() -> u64 { + 10 +} + +#[derive(Debug, Deserialize, Serialize, Clone, PartialEq)] +#[serde(deny_unknown_fields)] +pub struct AccessRule { + // A label for the Users page and the logs, e.g. "PCD operations". + pub name: String, + + // Group CNs ("DL-pcol2operations") or full DNs. A CN matches any group + // with that CN; a DN only that group. Case-insensitive, as the directory is. + #[serde(default)] + pub groups: Vec, + + // Usernames, for the odd person who needs access without a group. + #[serde(default)] + pub users: Vec, + + // Restricted unless written: a rule that forgets `role:` must never hand + // out admin, the same default api_keys.yaml has. + #[serde(default)] + pub role: ApiKeyRole, + + #[serde(default)] + pub sources: Vec, + + #[serde(default)] + pub endpoints: Vec, +} + +// What the rules grant one person. Kept free of the HTTP layer's Permissions +// type (domain does not depend on adapters); the login handler converts. +#[derive(Debug, Clone, PartialEq)] +pub enum Grant { + Admin, + Scoped { + sources: HashSet, + endpoints: HashSet, + }, +} + +// The CN of a DN: "CN=DL-ops,OU=Groups,DC=x" → "DL-ops". Escaped commas +// ("CN=Smith\, Jo,...") stay part of the value. +fn common_name(dn: &str) -> Option<&str> { + let first = split_rdn(dn); + let (attr, value) = first.split_once('=')?; + attr.trim() + .eq_ignore_ascii_case("cn") + .then_some(value.trim()) +} + +fn split_rdn(dn: &str) -> &str { + let mut escaped = false; + for (index, ch) in dn.char_indices() { + match ch { + '\\' if !escaped => escaped = true, + ',' if !escaped => return &dn[..index], + _ => escaped = false, + } + } + dn +} + +fn group_matches(wanted: &str, member_of: &[String]) -> bool { + if wanted.contains('=') { + member_of.iter().any(|dn| dn.eq_ignore_ascii_case(wanted)) + } else { + member_of + .iter() + .filter_map(|dn| common_name(dn)) + .any(|cn| cn.eq_ignore_ascii_case(wanted)) + } +} + +// The rules applied to one authenticated person. None = no rule names them. +pub fn grant_for(rules: &[AccessRule], username: &str, member_of: &[String]) -> Option { + let mut matched = false; + let mut sources = HashSet::new(); + let mut endpoints = HashSet::new(); + + for rule in rules { + let by_user = rule.users.iter().any(|u| u.eq_ignore_ascii_case(username)); + let by_group = rule.groups.iter().any(|g| group_matches(g, member_of)); + if !(by_user || by_group) { + continue; + } + if rule.role == ApiKeyRole::Admin { + return Some(Grant::Admin); + } + matched = true; + sources.extend(rule.sources.iter().cloned()); + endpoints.extend(rule.endpoints.iter().cloned()); + } + + matched.then_some(Grant::Scoped { sources, endpoints }) +} + +// The user filter with the username substituted, escaped so `*`, `(` or `\` +// in a username are literals rather than filter syntax: without it, typing +// `*` as a username would match the first account the search returns. +pub fn user_filter(template: &str, username: &str) -> String { + template.replace("{username}", &escape_filter_value(username)) +} + +// RFC 4515: the five characters with a meaning inside a filter value become +// \XX. Written here rather than borrowed from the LDAP client so the domain +// keeps no dependency on an adapter's crate. +fn escape_filter_value(value: &str) -> String { + let mut out = String::with_capacity(value.len()); + for ch in value.chars() { + match ch { + '*' => out.push_str("\\2a"), + '(' => out.push_str("\\28"), + ')' => out.push_str("\\29"), + '\\' => out.push_str("\\5c"), + '\0' => out.push_str("\\00"), + other => out.push(other), + } + } + out +} + +#[cfg(test)] +mod tests { + use super::*; + + fn rule(name: &str, role: ApiKeyRole) -> AccessRule { + AccessRule { + name: name.to_string(), + groups: Vec::new(), + users: Vec::new(), + role, + sources: Vec::new(), + endpoints: Vec::new(), + } + } + + const OPS: &str = "CN=DL-pcol2operations,OU=Groups,DC=example,DC=com"; + + #[test] + fn a_group_cn_matches_the_users_group_dn() { + let mut admins = rule("ops", ApiKeyRole::Admin); + admins.groups = vec!["dl-PCOL2operations".to_string()]; + assert_eq!( + grant_for(&[admins], "jdoe", &[OPS.to_string()]), + Some(Grant::Admin) + ); + } + + #[test] + fn a_full_dn_matches_only_that_group() { + let mut admins = rule("ops", ApiKeyRole::Admin); + admins.groups = vec!["CN=DL-pcol2operations,OU=Other,DC=example,DC=com".to_string()]; + assert_eq!(grant_for(&[admins], "jdoe", &[OPS.to_string()]), None); + } + + #[test] + fn nobody_named_by_a_rule_gets_nothing() { + let mut admins = rule("ops", ApiKeyRole::Admin); + admins.groups = vec!["DL-other".to_string()]; + assert_eq!(grant_for(&[admins], "jdoe", &[OPS.to_string()]), None); + } + + #[test] + fn scoped_rules_union_and_admin_wins() { + let mut vmware = rule("vmware", ApiKeyRole::Restricted); + vmware.users = vec!["JDoe".to_string()]; + vmware.sources = vec!["src-vmware".to_string()]; + let mut d42 = rule("d42", ApiKeyRole::Restricted); + d42.groups = vec!["DL-pcol2operations".to_string()]; + d42.sources = vec!["src-d42".to_string()]; + d42.endpoints = vec!["ep-awx".to_string()]; + + let grant = grant_for(&[vmware.clone(), d42.clone()], "jdoe", &[OPS.to_string()]); + let Some(Grant::Scoped { sources, endpoints }) = grant else { + panic!("expected a scoped grant, got {grant:?}"); + }; + assert!(sources.contains("src-vmware") && sources.contains("src-d42")); + assert!(endpoints.contains("ep-awx")); + + let mut admins = rule("admins", ApiKeyRole::Admin); + admins.users = vec!["jdoe".to_string()]; + assert_eq!( + grant_for(&[vmware, admins, d42], "jdoe", &[]), + Some(Grant::Admin) + ); + } + + #[test] + fn an_escaped_comma_stays_inside_the_cn() { + assert_eq!( + common_name("CN=Ops\\, Spain,OU=Groups,DC=x"), + Some("Ops\\, Spain") + ); + assert_eq!(common_name("OU=Groups,DC=x"), None); + } + + #[test] + fn filter_syntax_in_a_username_is_escaped() { + assert_eq!( + user_filter("(sAMAccountName={username})", "*)(uid=*"), + "(sAMAccountName=\\2a\\29\\28uid=\\2a)" + ); + } +} diff --git a/src/domain/mod.rs b/src/domain/mod.rs index daf58a1..9399674 100644 --- a/src/domain/mod.rs +++ b/src/domain/mod.rs @@ -6,11 +6,13 @@ pub mod endpoint; pub mod enricher; pub mod group_trim; pub mod host_filter; +pub mod ldap; pub mod project; pub mod source; pub mod static_inventory; pub mod sync_health; pub mod sync_mode; +pub mod topology; pub mod var_path; pub mod view; pub mod yaml; diff --git a/src/domain/topology.rs b/src/domain/topology.rs new file mode 100644 index 0000000..4f2bf21 --- /dev/null +++ b/src/domain/topology.rs @@ -0,0 +1,221 @@ +use serde::{Deserialize, Serialize}; +use std::collections::{BTreeMap, HashMap}; +use utoipa::ToSchema; + +use crate::domain::source::{ConnectorType, Source}; + +// The federation as seen from one instance: which other unified-api +// instances it pulls from (its peers), through which sources, and — asked of +// each peer in turn — what those pull from. A central's topology is the map +// of its edges; an edge's is itself alone. +// +// The same shape at every level, so a peer's answer nests as-is under the +// peer that asked for it. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, ToSchema)] +pub struct Topology { + /// The running version, as the instance reports it + pub version: String, + /// Sources configured on the instance. Absent when the peer is too old to + /// describe itself, and only its version could be read. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub sources: Option, + /// The instances it pulls from, by base URL + #[serde(default)] + pub peers: Vec, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, ToSchema)] +pub struct Peer { + /// The peer's base URL, as configured on the sources that read from it + pub url: String, + /// The local sources that read from this peer + pub sources: Vec, + /// What the peer says about itself, when it could be asked. Absent with + /// `error` set when it could not, or when the depth limit was reached. + #[serde(default, skip_serializing_if = "Option::is_none")] + #[schema(no_recursion)] + pub instance: Option>, + /// Why `instance` is missing: unreachable, refused, too deep + #[serde(default, skip_serializing_if = "Option::is_none")] + pub error: Option, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, ToSchema)] +pub struct PeerSource { + /// The source id on this instance + pub source_id: String, + /// The source it reads on the peer + pub remote_source: String, + pub cached: bool, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub hosts: Option, + /// Groups in that cache; null when nothing is cached + #[serde(default)] + pub groups: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub age_seconds: Option, + /// Failed pulls since the last success — 0 while healthy + pub consecutive_failures: u32, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub last_error: Option, +} + +// The peers of a configuration: every remote source the caller may see, +// grouped by the URL it reads from (trailing slashes ignored, so two spellings +// of one instance are one peer). Sorted by URL, then source id, so the answer +// is stable. The per-source state (cache, health) is filled in by the caller. +// The instance a remote source pulls from, as one spelling: a trailing slash +// would otherwise make the same peer two. None for any other connector, and +// for a remote one missing its `url` (config validation names that). +pub fn remote_url(source: &Source) -> Option { + if source.connector_type != ConnectorType::Remote { + return None; + } + source + .config + .get("url") + .map(|url| without_userinfo(url.trim_end_matches('/'))) +} + +// "https://user:pass@edge:8443" -> "https://edge:8443". An address shown back +// (a remote source's on its page and the topology, a project's git_url) must +// not carry credentials someone wrote into it. What reads the address to +// connect (the remote connector, git) uses the configured value, unaffected. +pub fn without_userinfo(url: &str) -> String { + let Some(scheme_end) = url.find("://") else { + return url.to_string(); + }; + let rest = &url[scheme_end + 3..]; + let authority_end = rest.find('/').unwrap_or(rest.len()); + match rest[..authority_end].rfind('@') { + Some(at) => format!("{}{}", &url[..scheme_end + 3], &rest[at + 1..]), + None => url.to_string(), + } +} + +pub fn peers_of( + sources: &HashMap, + visible: impl Fn(&str) -> bool, +) -> BTreeMap> { + let mut peers: BTreeMap> = BTreeMap::new(); + for (id, source) in sources { + if !visible(id) { + continue; + } + let Some(url) = remote_url(source) else { + continue; + }; + peers.entry(url).or_default().push((id.as_str(), source)); + } + for members in peers.values_mut() { + members.sort_by_key(|(id, _)| *id); + } + peers +} + +#[cfg(test)] +mod tests { + use super::*; + + fn source(connector: &str, url: Option<&str>) -> Source { + let mut yaml = format!( + "name: x\nproject_id: p\nscript_path: src-remote\nttl_seconds: 60\nconnector_type: {connector}\n" + ); + if let Some(url) = url { + yaml.push_str(&format!("config:\n url: \"{url}\"\n")); + } + serde_saphyr::from_str(&yaml).unwrap() + } + + #[test] + fn credentials_in_a_remote_url_are_never_shown() { + assert_eq!( + remote_url(&source( + "remote", + Some("https://svc:s3cret@edge.example:8443/") + )) + .as_deref(), + Some("https://edge.example:8443") + ); + // An "@" after the host is part of the path, not credentials. + assert_eq!( + without_userinfo("https://edge.example/a@b"), + "https://edge.example/a@b" + ); + } + + #[test] + fn remote_url_is_only_a_remote_sources_and_has_one_spelling() { + assert_eq!( + remote_url(&source("remote", Some("https://dc1.example:8443/"))).as_deref(), + Some("https://dc1.example:8443") + ); + assert_eq!(remote_url(&source("remote", None)), None); + // A script may carry a `url` of its own; it is not a peer. + assert_eq!( + remote_url(&source("script", Some("https://api.example"))), + None + ); + } + + #[test] + fn remote_sources_group_by_peer_url() { + let sources: HashMap = [ + ( + "src-dc1-facts", + source("remote", Some("https://dc1.example:8443")), + ), + ( + "src-dc1-kvm", + source("remote", Some("https://dc1.example:8443/")), + ), + ( + "src-dc2-facts", + source("remote", Some("https://dc2.example:8443")), + ), + ("src-d42", source("script", None)), + ] + .into_iter() + .map(|(id, s)| (id.to_string(), s)) + .collect(); + + let peers = peers_of(&sources, |_| true); + let urls: Vec<&String> = peers.keys().collect(); + assert_eq!( + urls, + ["https://dc1.example:8443", "https://dc2.example:8443"] + ); + let dc1: Vec<&str> = peers["https://dc1.example:8443"] + .iter() + .map(|(id, _)| *id) + .collect(); + assert_eq!(dc1, ["src-dc1-facts", "src-dc1-kvm"]); + } + + // A restricted caller sees the peers of the sources it may read, and no + // others: the URLs of the rest are not its business. + #[test] + fn a_caller_sees_only_peers_behind_its_sources() { + let sources: HashMap = [ + ("src-dc1", source("remote", Some("https://dc1.example"))), + ("src-dc2", source("remote", Some("https://dc2.example"))), + ] + .into_iter() + .map(|(id, s)| (id.to_string(), s)) + .collect(); + + let peers = peers_of(&sources, |id| id == "src-dc1"); + assert_eq!(peers.keys().collect::>(), ["https://dc1.example"]); + } + + // A peer's answer is the same shape, so it nests without translation. + #[test] + fn a_peer_answer_nests_as_is() { + let edge = r#"{"version":"0.33.0","sources":3,"peers":[]}"#; + let topology: Topology = serde_json::from_str(edge).unwrap(); + assert_eq!(topology.sources, Some(3)); + let old = r#"{"version":"0.32.0"}"#; + let topology: Topology = serde_json::from_str(old).unwrap(); + assert!(topology.peers.is_empty() && topology.sources.is_none()); + } +} diff --git a/src/lib.rs b/src/lib.rs index 0bbb541..9c50b4c 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -54,6 +54,8 @@ pub struct AppBuilder { config_store: Option>, live_settings: Option, api_keys: Vec, + directory: Arc, + peers: Arc, project_health: Arc, cors_allowed_origins: Vec, readyz_require_all_sources: bool, @@ -81,6 +83,10 @@ impl AppBuilder { config_store: None, live_settings: None, api_keys: Vec::new(), + // The real adapter even in tests: it is only called when a test + // configures LDAP, and those tests swap in a double. + directory: Arc::new(adapters::out::directory::ldap::LdapDirectory), + peers: Arc::new(adapters::out::peers::http::HttpPeers::new()), project_health: Arc::new(domain::sync_health::SyncHealthRegistry::new()), cors_allowed_origins: Vec::new(), readyz_require_all_sources: false, @@ -133,6 +139,16 @@ impl AppBuilder { self } + pub fn directory(mut self, directory: Arc) -> Self { + self.directory = directory; + self + } + + pub fn peers(mut self, peers: Arc) -> Self { + self.peers = peers; + self + } + pub fn secrets(mut self, secrets: Arc) -> Self { self.secrets = secrets; self @@ -175,6 +191,9 @@ impl AppBuilder { self.refresh_max_concurrent = cfg.server.refresh_max_concurrent; self.shutdown_grace_seconds = cfg.server.shutdown_grace_seconds; self.max_body_bytes = cfg.server.max_body_bytes; + // What this process is built from, kept whether or not the config + // API is on: GET /settings shows the restart-only values in force. + self.live_settings = Some(config::RestartOnlySettings::from_config(cfg)); self } @@ -289,6 +308,8 @@ impl AppBuilder { shutdown_grace_seconds: self.shutdown_grace_seconds, refresh_timeout_seconds: self.refresh_timeout_seconds, refresh_max_concurrent: self.refresh_max_concurrent, + ldap: self.ui.ldap.clone(), + ui_home: self.ui.home().to_string(), })), config_store: self.config_store, live_settings: std::sync::RwLock::new(self.live_settings), @@ -297,11 +318,15 @@ impl AppBuilder { projects_dir: self.projects_dir, ui: self.ui, sessions: std::sync::Arc::new(crate::adapters::r#in::http::session::SessionStore::new()), + directory: self.directory, + peers: self.peers, sync_health: Arc::new(domain::sync_health::SyncHealthRegistry::new()), advertised_scopes: Arc::new(domain::source::AdvertisedScopeRegistry::new()), enrich_health: Arc::new(domain::sync_health::SyncHealthRegistry::new()), project_health: self.project_health, snapshot_health: Arc::new(domain::sync_health::SyncHealthRegistry::new()), + started: std::time::Instant::now(), + login_throttle: Arc::new(crate::application::login::LoginThrottle::default()), refresh: Arc::new(application::refresh::RefreshCoordinator::new( self.refresh_max_concurrent, self.refresh_timeout_seconds, diff --git a/src/main.rs b/src/main.rs index 1457335..114dc67 100644 --- a/src/main.rs +++ b/src/main.rs @@ -68,6 +68,11 @@ async fn main() { } }; + if let Err(e) = unified_api::adapters::r#in::http::auth::ldap_requires_keys(&cfg, &api_keys) { + error!("Refusing to start: {}", e); + std::process::exit(1); + } + // Make an unauthenticated deployment loud, not a buried auth=false field if api_keys.is_empty() { warn!( diff --git a/src/ports/directory.rs b/src/ports/directory.rs new file mode 100644 index 0000000..311aa54 --- /dev/null +++ b/src/ports/directory.rs @@ -0,0 +1,45 @@ +use std::future::Future; +use std::pin::Pin; + +use crate::domain::ldap::LdapConfig; + +// DirectoryPort — check a username and password against a user directory +// (LDAP / Active Directory) and say which groups the person belongs to. +// +// It answers who someone is, never what they may do: access is decided by +// the `access` rules in the configuration (domain::ldap::grant_for), so the +// port has no notion of roles at all. +pub trait DirectoryPort: Send + Sync { + fn authenticate<'a>(&'a self, request: DirectoryLogin<'a>) -> DirectoryFuture<'a>; +} + +pub type DirectoryFuture<'a> = + Pin> + Send + 'a>>; + +pub struct DirectoryLogin<'a> { + pub config: &'a LdapConfig, + // The service account that searches for the user, resolved from + // `bind_credential` by the caller. + pub bind_dn: &'a str, + pub bind_password: &'a str, + pub username: &'a str, + pub password: &'a str, +} + +#[derive(Debug, Clone)] +pub struct DirectoryUser { + pub dn: String, + // The account's own name (`username_attribute`), when the entry has it. + pub username: Option, + pub member_of: Vec, +} + +#[derive(Debug, PartialEq)] +pub enum DirectoryError { + // Unknown user or wrong password — one variant for both, because the + // difference is exactly what someone guessing usernames wants to learn. + InvalidCredentials, + // The directory could not answer: unreachable, TLS refused, the service + // account rejected, a timeout. The message is for the log, not the user. + Unavailable(String), +} diff --git a/src/ports/mod.rs b/src/ports/mod.rs index 1eeb890..ce039e6 100644 --- a/src/ports/mod.rs +++ b/src/ports/mod.rs @@ -1,8 +1,10 @@ pub mod cache; pub mod config_store; pub mod connector; +pub mod directory; pub mod enricher; pub mod git; pub mod output; +pub mod peer; pub mod secrets; pub mod venv; diff --git a/src/ports/peer.rs b/src/ports/peer.rs new file mode 100644 index 0000000..5430255 --- /dev/null +++ b/src/ports/peer.rs @@ -0,0 +1,24 @@ +use std::future::Future; +use std::pin::Pin; + +use crate::domain::topology::Topology; + +// PeerPort — ask another unified-api instance to describe itself, for the +// federation map (GET /api/v1/topology). Read-only, and never on the data +// path: a peer that cannot be asked is shown as such, nothing fails. +pub trait PeerPort: Send + Sync { + fn describe(&self, request: PeerRequest) -> PeerFuture; +} + +pub type PeerFuture = Pin> + Send>>; + +// Owned, so the requests to every peer can run concurrently on their own tasks. +pub struct PeerRequest { + pub url: String, + // The API key the sources reading from this peer already use. + pub api_key: Option, + pub insecure_tls: bool, + pub timeout_seconds: u64, + // How many further hops the peer may take asking its own peers. + pub depth: u8, +} diff --git a/src/state.rs b/src/state.rs index 8cedf6e..689583b 100644 --- a/src/state.rs +++ b/src/state.rs @@ -63,6 +63,12 @@ pub struct RuntimeConfig { // then re-applies onto it (application::config::apply calls resize). pub refresh_timeout_seconds: u64, pub refresh_max_concurrent: usize, + // LDAP login for the browsable API. Read per login, so the directory and + // the access rules reload; a change also ends the sessions opened under + // the old rules (application::config::apply). + pub ldap: Option, + // `server.ui.home`: where a sign-in with no page to return to lands. + pub ui_home: String, } // Bumped on every reload, and watched by the scheduler: the tasks of the @@ -157,6 +163,11 @@ pub struct AppState { // `ui.enabled` is false — the routes that would create one are not in the // router at all. pub sessions: Arc, + // Checks LDAP logins. Unused unless `server.ui.ldap` is configured. + pub directory: Arc, + // Asks other unified-api instances to describe themselves, for the + // federation map. Read-only and never on the data path. + pub peers: Arc, // The browsable HTML API's settings. Static like `projects_dir`, and for // the same reason: it decides which middlewares and routes are in the @@ -179,6 +190,12 @@ pub struct AppState { pub enrich_health: Arc, pub project_health: Arc, pub snapshot_health: Arc, + // When this process's app was built: how long it has been running, for + // the Configuration page ("did it restart?"). + pub started: std::time::Instant, + // Recent failed LDAP logins and the cap on concurrent ones (see + // application::login::LoginThrottle). + pub login_throttle: Arc, // Coalescing and limits for reads that are allowed to refresh before // answering. In-process state like sync_health: no outside world behind it. pub refresh: Arc, diff --git a/tests/credentials_test.rs b/tests/credentials_test.rs new file mode 100644 index 0000000..bcc2c8d --- /dev/null +++ b/tests/credentials_test.rs @@ -0,0 +1,143 @@ +use axum::body::Body; +use axum::http::{Request, StatusCode}; +use http_body_util::BodyExt; +use std::collections::{HashMap, HashSet}; +use std::sync::Arc; +use tower::ServiceExt; + +use unified_api::adapters::r#in::http::auth::{Permissions, ResolvedApiKey}; +use unified_api::adapters::out::secrets::env::EnvSecrets; +use unified_api::domain::credential::Credential; +use unified_api::domain::source::Source; + +// The credentials route, through the real router and the real env resolver: +// it says where each secret lives, whether it resolves, and what uses it — +// and never, anywhere in its body, a secret's value. + +const SECRET: &str = "s3cr3t-value-that-must-never-appear"; + +fn credential(yaml: &str) -> Credential { + serde_saphyr::from_str(yaml).unwrap() +} + +fn app() -> axum::Router { + // A name no other test uses, so setting it cannot race another test. + // SAFETY: set before the app is built, read only by this test's resolver. + unsafe { std::env::set_var("CREDPAGE_OK_TOKEN", SECRET) }; + let credentials: HashMap = [ + ( + "cred-ok".to_string(), + credential("name: Works\ntype: token\nenv_prefix: CREDPAGE_OK\nsecret_keys:\n token: TOKEN\n"), + ), + ( + "cred-missing".to_string(), + credential( + "name: Missing\ntype: username_password\nenv_prefix: CREDPAGE_MISSING\nsecret_keys:\n username: USER\n password: PASS\n", + ), + ), + ] + .into_iter() + .collect(); + let source: Source = serde_saphyr::from_str( + "name: x\nproject_id: p\nscript_path: s.py\nttl_seconds: 60\ncredential_ids: [cred-ok]\n", + ) + .unwrap(); + + unified_api::AppBuilder::new() + .sources([("src-a".to_string(), source)].into_iter().collect()) + .secrets(Arc::new(EnvSecrets::new(credentials.clone()))) + .credentials(credentials, Default::default()) + .api_keys(vec![ + ResolvedApiKey { + name: "admin".to_string(), + secret: "adm".to_string(), + permissions: Permissions::Admin, + }, + ResolvedApiKey { + name: "awx".to_string(), + secret: "awx".to_string(), + permissions: Permissions::Scoped { + sources: HashSet::new(), + endpoints: HashSet::new(), + }, + }, + ]) + .build() +} + +async fn get(app: axum::Router, key: &str) -> (StatusCode, String) { + let resp = app + .oneshot( + Request::get("/api/v1/credentials") + .header("x-api-key", key) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + let status = resp.status(); + let body = resp.into_body().collect().await.unwrap().to_bytes(); + (status, String::from_utf8(body.to_vec()).unwrap()) +} + +#[tokio::test] +async fn credentials_say_where_and_whether_but_never_what() { + let (status, body) = get(app(), "adm").await; + assert_eq!(status, StatusCode::OK, "{body}"); + assert!(!body.contains(SECRET), "a secret value leaked: {body}"); + + let list: Vec = serde_json::from_str(&body).unwrap(); + let by_id = |id: &str| list.iter().find(|c| c["credential_id"] == id).unwrap(); + + let ok = by_id("cred-ok"); + assert_eq!(ok["resolves"], true); + assert_eq!(ok["backend"], "env"); + assert_eq!(ok["env_vars"], serde_json::json!(["CREDPAGE_OK_TOKEN"])); + assert_eq!(ok["used_by"]["sources"], serde_json::json!(["src-a"])); + + // Missing: says so, and names what is missing rather than failing blank. + let missing = by_id("cred-missing"); + assert_eq!(missing["resolves"], false); + assert!( + missing["error"] + .as_str() + .unwrap() + .contains("CREDPAGE_MISSING_"), + "{missing}" + ); + assert_eq!(missing["used_by"]["sources"], serde_json::json!([])); +} + +#[tokio::test] +async fn credentials_are_admin_only() { + let (status, _) = get(app(), "awx").await; + assert_eq!(status, StatusCode::FORBIDDEN); +} + +// The configuration in force: defaults included (what config.yaml leaves +// out is still answered), admin-only like the configuration API. +#[tokio::test] +async fn settings_show_values_in_force_to_admins_only() { + let request = |key: &'static str| { + app().oneshot( + Request::get("/api/v1/settings") + .header("x-api-key", key) + .body(Body::empty()) + .unwrap(), + ) + }; + let resp = request("adm").await.unwrap(); + assert_eq!(resp.status(), StatusCode::OK); + let body = resp.into_body().collect().await.unwrap().to_bytes(); + let settings: serde_json::Value = serde_json::from_slice(&body).unwrap(); + // Set nowhere in this app, so these are the defaults. + assert_eq!(settings["server"]["metrics_require_auth"], false); + assert_eq!(settings["server"]["ui"]["session_ttl_seconds"], 3600); + assert_eq!(settings["secrets"]["vault"], serde_json::Value::Null); + assert_eq!(settings["restart_pending"], serde_json::json!([])); + + assert_eq!( + request("awx").await.unwrap().status(), + StatusCode::FORBIDDEN + ); +} diff --git a/tests/git_test.rs b/tests/git_test.rs index 69ebdf0..2d870da 100644 --- a/tests/git_test.rs +++ b/tests/git_test.rs @@ -243,3 +243,39 @@ async fn project_sync_unknown_id_is_404_and_bad_repo_is_502() { assert_eq!(status, StatusCode::BAD_GATEWAY); assert!(body.contains("\"success\":false")); } + +// What reads its scripts from a checkout: the list names them, so a project +// can be seen to be unused (safe to remove) or relied on. +#[tokio::test] +async fn a_project_lists_what_uses_it() { + let workdir = tempfile::tempdir().unwrap(); + let projects = [ + ("prj-test".to_string(), project("file:///nowhere/repo.git")), + ("prj-idle".to_string(), project("file:///nowhere/idle.git")), + ] + .into_iter() + .collect(); + let source: unified_api::domain::source::Source = serde_saphyr::from_str( + "name: x\nproject_id: prj-test\nscript_path: fetch.py\nttl_seconds: 60\n", + ) + .unwrap(); + let app = unified_api::AppBuilder::new() + .sources([("src-a".to_string(), source)].into_iter().collect()) + .projects(projects, workdir.path().to_path_buf()) + .api_keys(keys()) + .build(); + + let (status, body) = request(app, "GET", "/api/v1/projects", "adm").await; + assert_eq!(status, StatusCode::OK); + let list: Vec = serde_json::from_str(&body).unwrap(); + let by_id = |id: &str| list.iter().find(|p| p["project_id"] == id).unwrap().clone(); + assert_eq!( + by_id("prj-test")["used_by"]["sources"], + serde_json::json!(["src-a"]) + ); + assert_eq!( + by_id("prj-idle")["used_by"]["sources"], + serde_json::json!([]) + ); + assert_eq!(by_id("prj-test")["timeout_seconds"], 300); +} diff --git a/tests/ldap_login_test.rs b/tests/ldap_login_test.rs new file mode 100644 index 0000000..a97e583 --- /dev/null +++ b/tests/ldap_login_test.rs @@ -0,0 +1,539 @@ +use axum::body::Body; +use axum::http::{Request, StatusCode, header}; +use std::collections::HashMap; +use std::sync::Arc; +use tower::ServiceExt; + +use unified_api::adapters::r#in::http::auth::{Permissions, ResolvedApiKey}; +use unified_api::adapters::out::secrets::mock::MockSecrets; +use unified_api::domain::api_key::ApiKeyRole; +use unified_api::domain::ldap::{AccessRule, LdapConfig}; +use unified_api::ports::directory::{ + DirectoryError, DirectoryFuture, DirectoryLogin, DirectoryPort, DirectoryUser, +}; + +// LDAP login, end to end through the real router: the form, the session it +// opens, and what that session may do. The directory is a double; the +// adapter's own behaviour (empty passwords, unreachable servers) is tested +// next to it. + +// Knows jdoe (in DL-ops) and asmith (in DL-readers), both with "right". +struct Directory; + +impl DirectoryPort for Directory { + fn authenticate<'a>(&'a self, request: DirectoryLogin<'a>) -> DirectoryFuture<'a> { + Box::pin(async move { + let group = match request.username { + "jdoe" => "CN=DL-ops,OU=Groups,DC=x", + "asmith" => "CN=DL-readers,OU=Groups,DC=x", + "nobody" => "CN=DL-elsewhere,OU=Groups,DC=x", + _ => return Err(DirectoryError::InvalidCredentials), + }; + if request.password != "right" { + return Err(DirectoryError::InvalidCredentials); + } + Ok(DirectoryUser { + dn: format!("CN={},OU=Users,DC=x", request.username), + username: Some(request.username.to_string()), + member_of: vec![group.to_string()], + }) + }) + } +} + +fn ldap() -> LdapConfig { + let rule = |name: &str, group: &str, role: ApiKeyRole, sources: &[&str]| AccessRule { + name: name.to_string(), + groups: vec![group.to_string()], + users: Vec::new(), + role, + sources: sources.iter().map(|s| s.to_string()).collect(), + endpoints: Vec::new(), + }; + LdapConfig { + url: "ldaps://ldap.example.com".to_string(), + starttls: false, + ca_file: None, + bind_credential: "cred-ldap".to_string(), + search_base: "DC=x".to_string(), + user_filter: "(sAMAccountName={username})".to_string(), + group_attribute: "memberOf".to_string(), + username_attribute: "sAMAccountName".to_string(), + timeout_seconds: 5, + access: vec![ + rule("Operations", "DL-ops", ApiKeyRole::Admin, &[]), + rule( + "Readers", + "DL-readers", + ApiKeyRole::Restricted, + &["src-vmware"], + ), + ], + } +} + +fn app(ldap: Option) -> axum::Router { + app_home(ldap, "/api/v1/") +} + +fn app_home(ldap: Option, home: &str) -> axum::Router { + let ui = unified_api::config::UiConfig { + enabled: true, + ldap, + home: Some(home.to_string()), + ..Default::default() + }; + unified_api::AppBuilder::new() + .sources(HashMap::new()) + .ui(ui) + .secrets(Arc::new(MockSecrets::new().with( + "cred-ldap", + [("username", "CN=svc-unified,DC=x"), ("password", "svc")], + ))) + .directory(Arc::new(Directory)) + // With keys configured the API is not open, so a session is the only + // way these requests authenticate. + .api_keys(vec![ResolvedApiKey { + name: "AWX".to_string(), + secret: "awx-secret".to_string(), + permissions: Permissions::Admin, + }]) + .build() +} + +async fn body(resp: axum::response::Response) -> String { + let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX) + .await + .unwrap(); + String::from_utf8_lossy(&bytes).to_string() +} + +fn cookie(resp: &axum::response::Response, name: &str) -> Option { + resp.headers() + .get_all(header::SET_COOKIE) + .iter() + .filter_map(|v| v.to_str().ok()) + .find_map(|c| { + let pair = c.split(';').next()?; + let (key, value) = pair.split_once('=')?; + (key == name && !value.is_empty()).then(|| value.to_string()) + }) +} + +// GET the form (which plants the login challenge), then POST it back with +// `fields`, the way a browser does. +async fn submit(app: &axum::Router, fields: &str) -> axum::response::Response { + let form = app + .clone() + .oneshot(Request::get("/api/v1/login").body(Body::empty()).unwrap()) + .await + .unwrap(); + let challenge = cookie(&form, "uapi_login").expect("login challenge cookie"); + + app.clone() + .oneshot( + Request::post("/api/v1/login") + .header(header::CONTENT_TYPE, "application/x-www-form-urlencoded") + .header(header::COOKIE, format!("uapi_login={challenge}")) + .body(Body::from(format!("_csrf={challenge}&{fields}"))) + .unwrap(), + ) + .await + .unwrap() +} + +async fn get_json(app: &axum::Router, uri: &str, session: &str) -> (StatusCode, serde_json::Value) { + let resp = app + .clone() + .oneshot( + Request::get(uri) + .header(header::COOKIE, format!("uapi_session={session}")) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + let status = resp.status(); + let text = body(resp).await; + ( + status, + serde_json::from_str(&text).unwrap_or(serde_json::Value::Null), + ) +} + +#[tokio::test] +async fn the_form_asks_for_a_username_only_when_ldap_is_configured() { + let with = body( + app(Some(ldap())) + .oneshot(Request::get("/api/v1/login").body(Body::empty()).unwrap()) + .await + .unwrap(), + ) + .await; + assert!(with.contains(r#"name="username""#)); + assert!( + with.contains(r#"name="key""#), + "the API key stays available" + ); + + let without = body( + app(None) + .oneshot(Request::get("/api/v1/login").body(Body::empty()).unwrap()) + .await + .unwrap(), + ) + .await; + assert!(!without.contains(r#"name="username""#)); +} + +#[tokio::test] +async fn an_admin_group_member_gets_an_admin_session() { + let app = app(Some(ldap())); + let resp = submit(&app, "username=jdoe&password=right").await; + assert_eq!(resp.status(), StatusCode::SEE_OTHER); + let session = cookie(&resp, "uapi_session").expect("a session cookie"); + + let (status, auth) = get_json(&app, "/api/v1/auth", &session).await; + assert_eq!(status, StatusCode::OK); + assert_eq!(auth["me"]["name"], "ldap:jdoe"); + assert_eq!(auth["me"]["method"], "ldap"); + assert_eq!(auth["me"]["role"], "admin"); + assert_eq!(auth["methods"]["ldap"], true); + // An admin sees the configuration and the keys, never a secret + assert_eq!(auth["ldap"]["bind_credential"], "cred-ldap"); + assert_eq!(auth["ldap"]["access"][0]["name"], "Operations"); + assert_eq!(auth["api_keys"][0]["name"], "AWX"); + assert!(!auth.to_string().contains("awx-secret")); + assert!(!auth.to_string().contains("\"svc\"")); +} + +#[tokio::test] +async fn a_restricted_member_sees_only_themselves_and_their_scope() { + let app = app(Some(ldap())); + let resp = submit(&app, "username=asmith&password=right").await; + let session = cookie(&resp, "uapi_session").expect("a session cookie"); + + let (_, auth) = get_json(&app, "/api/v1/auth", &session).await; + assert_eq!(auth["me"]["role"], "restricted"); + assert_eq!(auth["me"]["sources"][0], "src-vmware"); + assert!(auth.get("ldap").is_none(), "the rules are an admin's map"); + assert!(auth.get("api_keys").is_none()); + + let (status, _) = get_json(&app, "/api/v1/projects", &session).await; + assert_eq!(status, StatusCode::FORBIDDEN); +} + +#[tokio::test] +async fn a_wrong_password_opens_nothing() { + let app = app(Some(ldap())); + let resp = submit(&app, "username=jdoe&password=wrong").await; + assert_eq!(resp.status(), StatusCode::UNAUTHORIZED); + assert!(cookie(&resp, "uapi_session").is_none()); + assert!(body(resp).await.contains("invalid username or password")); +} + +#[tokio::test] +async fn a_valid_account_no_rule_names_is_refused_and_told_why() { + let app = app(Some(ldap())); + let resp = submit(&app, "username=nobody&password=right").await; + assert_eq!(resp.status(), StatusCode::FORBIDDEN); + assert!(cookie(&resp, "uapi_session").is_none()); + assert!(body(resp).await.contains("no access to this instance")); +} + +#[tokio::test] +async fn an_api_key_still_logs_in_next_to_ldap() { + let app = app(Some(ldap())); + let resp = submit(&app, "key=awx-secret").await; + let session = cookie(&resp, "uapi_session").expect("a session cookie"); + let (_, auth) = get_json(&app, "/api/v1/auth", &session).await; + assert_eq!(auth["me"]["name"], "AWX"); + assert_eq!(auth["me"]["method"], "api_key"); +} + +#[tokio::test] +async fn a_username_on_an_instance_without_ldap_is_refused() { + let app = app(None); + let resp = submit(&app, "username=jdoe&password=right").await; + assert_eq!(resp.status(), StatusCode::BAD_REQUEST); + assert!(cookie(&resp, "uapi_session").is_none()); +} + +// After login the browser goes where the page that sent it asked — but only +// somewhere on this origin, or the login becomes an open redirect. +#[tokio::test] +async fn next_redirects_only_within_this_origin() { + let app = app(Some(ldap())); + let location = |resp: &axum::response::Response| { + resp.headers() + .get(header::LOCATION) + .and_then(|v| v.to_str().ok()) + .unwrap_or("") + .to_string() + }; + + let resp = submit(&app, "username=jdoe&password=right&next=/sources").await; + assert_eq!(location(&resp), "/sources"); + + for evil in [ + "//evil.example/x", + "https://evil.example", + "/\\evil.example", + // Browsers drop tabs and newlines: these become "//evil.example". + "/%09/evil.example", + "/%0A/evil.example", + "/%20/evil.example", + ] { + let resp = submit(&app, &format!("username=jdoe&password=right&next={evil}")).await; + assert_eq!(location(&resp), "/api/v1/", "followed {evil}"); + } +} + +// With nowhere asked for, a login lands on `server.ui.home`: "/" where the +// UI is served in front of the API, so signing in never strands a person on +// the API's own pages. +#[tokio::test] +async fn without_next_a_login_lands_on_the_configured_home() { + let location = |resp: &axum::response::Response| { + resp.headers() + .get(header::LOCATION) + .and_then(|v| v.to_str().ok()) + .unwrap_or("") + .to_string() + }; + let resp = submit(&app(Some(ldap())), "username=jdoe&password=right").await; + assert_eq!( + location(&resp), + "/api/v1/", + "the default, with no UI in front" + ); + + let resp = submit(&app_home(Some(ldap()), "/"), "username=jdoe&password=right").await; + assert_eq!(location(&resp), "/"); + // A page that asked still wins over home. + let resp = submit( + &app_home(Some(ldap()), "/"), + "username=jdoe&password=right&next=/sources", + ) + .await; + assert_eq!(location(&resp), "/sources"); +} + +// Signing out from the UI posts next=/, and the sign-in page it lands on +// carries it, so signing straight back in returns to the UI. +#[tokio::test] +async fn sign_out_keeps_where_the_next_sign_in_lands() { + let app = app(Some(ldap())); + let logout = |body: &'static str| { + let app = app.clone(); + async move { + let mut req = Request::post("/api/v1/logout"); + if !body.is_empty() { + req = req.header(header::CONTENT_TYPE, "application/x-www-form-urlencoded"); + } + let resp = app + .oneshot(req.body(Body::from(body)).unwrap()) + .await + .unwrap(); + resp.headers() + .get(header::LOCATION) + .and_then(|v| v.to_str().ok()) + .unwrap_or("") + .to_string() + } + }; + assert_eq!(logout("next=/").await, "/api/v1/login?next=/"); + assert_eq!( + logout("next=%2Fsources%3Fhost%3Da%26b").await, + "/api/v1/login?next=/sources%3Fhost%3Da%26b" + ); + assert_eq!(logout("next=//evil.example").await, "/api/v1/login"); + assert_eq!( + logout("").await, + "/api/v1/login", + "a bare POST still signs out" + ); +} + +// After a refusal the page must still be usable: it carries a fresh +// challenge, so correcting the password and submitting again works without +// reloading. +#[tokio::test] +async fn a_refused_form_can_be_submitted_again() { + let app = app(Some(ldap())); + let refused = submit(&app, "username=jdoe&password=wrong").await; + let challenge = cookie(&refused, "uapi_login").expect("a fresh challenge cookie"); + let page = body(refused).await; + assert!( + page.contains(&format!(r#"value="{challenge}""#)), + "the form carries it too" + ); + + let retry = app + .clone() + .oneshot( + Request::post("/api/v1/login") + .header(header::CONTENT_TYPE, "application/x-www-form-urlencoded") + .header(header::COOKIE, format!("uapi_login={challenge}")) + .body(Body::from(format!( + "_csrf={challenge}&username=jdoe&password=right" + ))) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(retry.status(), StatusCode::SEE_OTHER); + assert!(cookie(&retry, "uapi_session").is_some()); +} + +// One card, and it reopens on the way in that was just tried: a mistyped key +// must not drop the person back on the directory form. +#[tokio::test] +async fn the_card_reopens_on_the_method_that_was_tried() { + let app = app(Some(ldap())); + let page = body(submit(&app, "key=wrong").await).await; + assert!(page.contains(r#"id="mode-key" checked"#), "{page}"); + + let page = body(submit(&app, "username=jdoe&password=wrong").await).await; + assert!(page.contains(r#"id="mode-ldap" checked"#)); + assert_eq!(page.matches(r#"class="card""#).count(), 1); +} + +// The UI is a script, not a server-rendered form: it has nowhere to read the +// CSRF token from but `GET /auth`, and its writes only pass with it. +#[tokio::test] +async fn a_session_learns_its_csrf_token_from_auth_and_writes_with_it() { + let app = app(Some(ldap())); + let resp = submit(&app, "username=jdoe&password=right").await; + let session = cookie(&resp, "uapi_session").expect("a session cookie"); + + let (status, json) = get_json(&app, "/api/v1/auth", &session).await; + assert_eq!(status, StatusCode::OK); + let token = json["me"]["csrf_token"] + .as_str() + .expect("a session is given its token") + .to_string(); + + let sync = |csrf: Option<&str>| { + let mut request = Request::post("/api/v1/sources/src-unknown/sync") + .header(header::COOKIE, format!("uapi_session={session}")); + if let Some(csrf) = csrf { + request = request.header("x-csrf-token", csrf); + } + app.clone().oneshot(request.body(Body::empty()).unwrap()) + }; + assert_eq!(sync(None).await.unwrap().status(), StatusCode::FORBIDDEN); + // Past the CSRF check: refused only because the source does not exist. + assert_eq!( + sync(Some(&token)).await.unwrap().status(), + StatusCode::NOT_FOUND + ); +} + +#[tokio::test] +async fn a_header_caller_is_given_no_csrf_token() { + let resp = app(Some(ldap())) + .oneshot( + Request::get("/api/v1/auth") + .header("x-api-key", "awx-secret") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(resp.status(), StatusCode::OK); + let json: serde_json::Value = serde_json::from_str(&body(resp).await).unwrap(); + assert!(json["me"].get("csrf_token").is_none()); +} + +// Five wrong passwords and the form stops asking the directory about that +// account: a script cannot lock a real account out through this form, and +// the right password does not get in either until the window passes. Paused +// time: the fixed failure delay would otherwise make this test take seconds. +#[tokio::test(start_paused = true)] +async fn repeated_wrong_passwords_lock_the_form_not_the_account() { + let app = app(Some(ldap())); + for _ in 0..5 { + let resp = submit(&app, "username=jdoe&password=wrong").await; + assert_eq!(resp.status(), StatusCode::UNAUTHORIZED); + } + let resp = submit(&app, "username=JDoe&password=right").await; + assert_eq!( + resp.status(), + StatusCode::TOO_MANY_REQUESTS, + "case is not identity" + ); + assert!(cookie(&resp, "uapi_session").is_none()); + + // Another account is unaffected. + let resp = submit(&app, "username=asmith&password=right").await; + assert!(cookie(&resp, "uapi_session").is_some()); +} + +// A wrong password and an unknown user take the same time to answer, so +// timing cannot tell which usernames exist. +#[tokio::test(start_paused = true)] +async fn unknown_users_and_wrong_passwords_take_the_same_time() { + let app = app(Some(ldap())); + let timed = |fields: &'static str| { + let app = app.clone(); + async move { + let started = tokio::time::Instant::now(); + let resp = submit(&app, fields).await; + (resp.status(), started.elapsed()) + } + }; + let (unknown, t_unknown) = timed("username=nobody-at-all&password=x").await; + let (wrong, t_wrong) = timed("username=jdoe&password=x").await; + assert_eq!(unknown, StatusCode::UNAUTHORIZED); + assert_eq!(wrong, StatusCode::UNAUTHORIZED); + assert_eq!(t_unknown.as_millis() / 100, t_wrong.as_millis() / 100); +} + +// With /metrics behind authentication, a signed-in browser reads it with its +// session (the UI's Metrics page), and nobody else gets in without a key. +#[tokio::test] +async fn a_session_reads_protected_metrics() { + let ui = unified_api::config::UiConfig { + enabled: true, + ldap: Some(ldap()), + ..Default::default() + }; + let app = unified_api::AppBuilder::new() + .sources(HashMap::new()) + .ui(ui) + .metrics_require_auth(true) + .secrets(Arc::new(MockSecrets::new().with( + "cred-ldap", + [("username", "CN=svc-unified,DC=x"), ("password", "svc")], + ))) + .directory(Arc::new(Directory)) + .api_keys(vec![ResolvedApiKey { + name: "AWX".to_string(), + secret: "awx-secret".to_string(), + permissions: Permissions::Admin, + }]) + .build(); + let resp = submit(&app, "username=jdoe&password=right").await; + let session = cookie(&resp, "uapi_session").expect("a session cookie"); + + let scrape = |cookie: Option| { + let mut request = Request::get("/metrics"); + if let Some(c) = cookie { + request = request.header(header::COOKIE, format!("uapi_session={c}")); + } + app.clone().oneshot(request.body(Body::empty()).unwrap()) + }; + assert_eq!( + scrape(Some(session)).await.unwrap().status(), + StatusCode::OK + ); + assert_eq!( + scrape(None).await.unwrap().status(), + StatusCode::UNAUTHORIZED + ); + assert_eq!( + scrape(Some("forged".to_string())).await.unwrap().status(), + StatusCode::UNAUTHORIZED + ); +} diff --git a/tests/sync_test.rs b/tests/sync_test.rs index 11382f1..33c5f37 100644 --- a/tests/sync_test.rs +++ b/tests/sync_test.rs @@ -1257,6 +1257,15 @@ async fn list_endpoints_shows_readiness() { let eps: Vec = serde_json::from_str(&body).unwrap(); assert_eq!(eps[0]["sources_ready"], 0); assert_eq!(eps[0]["sources_missing"].as_array().unwrap().len(), 2); + // An open instance treats every caller as admin: the transformer shows, + // with the timeout a script actually gets. + let transform = &eps[0]["transform"]; + assert_eq!( + transform["script_path"], + "tests/adapters/out/output/ansible_inventory.py" + ); + assert_eq!(transform["output"], serde_json::Value::Null); + assert_eq!(transform["timeout_seconds"], 300); // Sync src-test let (_, _) = request(app.clone(), "POST", "/api/v1/sources/src-test/sync").await; @@ -2064,3 +2073,108 @@ async fn without_host_args_a_host_scoped_sync_calls_the_script_unchanged() { assert_eq!(group_hosts(&dataset, "legacy").len(), 2); assert!(dataset["groups"].get("modern").is_none()); } + +// A key granted an endpoint reads its output and readiness, not how it is +// made: the transformer is admin-only, like a source's gathering. +#[tokio::test] +async fn list_endpoints_hides_the_transform_from_a_restricted_key() { + use unified_api::adapters::r#in::http::auth::{Permissions, ResolvedApiKey}; + let mut endpoints = HashMap::new(); + endpoints.insert( + "ep-test".to_string(), + OutputEndpoint { + name: "Test Endpoint".to_string(), + source_ids: vec!["src-test".to_string()], + output: None, + script_path: Some("tests/adapters/out/output/ansible_inventory.py".to_string()), + script_args: vec![], + project_id: None, + config: HashMap::new(), + limit: None, + timeout_seconds: None, + }, + ); + let app = unified_api::AppBuilder::new() + .sources(HashMap::new()) + .endpoints(endpoints) + .api_keys(vec![ResolvedApiKey { + name: "awx".to_string(), + secret: "awx".to_string(), + permissions: Permissions::Scoped { + sources: std::collections::HashSet::new(), + endpoints: ["ep-test".to_string()].into_iter().collect(), + }, + }]) + .build(); + let resp = app + .oneshot( + Request::get("/api/v1/endpoints") + .header("x-api-key", "awx") + .body(axum::body::Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + let body = resp.into_body().collect().await.unwrap().to_bytes(); + let eps: Vec = serde_json::from_slice(&body).unwrap(); + assert_eq!(eps.len(), 1); + assert!(eps[0].get("transform").is_none()); +} + +// The counts are the size of what a call returns, not of the inputs: the +// same merge, so they must match the rendered output exactly. A script's +// output is its own business, so it gets no numbers rather than wrong ones. +#[tokio::test] +async fn endpoint_counts_match_the_default_output() { + use unified_api::domain::endpoint::OutputFormat; + let mut sources = HashMap::new(); + sources.insert("src-test".to_string(), test_source("default")); + let endpoint = |output: Option| OutputEndpoint { + name: "Test".to_string(), + source_ids: vec!["src-test".to_string()], + output, + script_path: output + .is_none() + .then(|| "tests/adapters/out/output/ansible_inventory.py".to_string()), + script_args: vec![], + project_id: None, + config: HashMap::new(), + limit: None, + timeout_seconds: None, + }; + let mut endpoints = HashMap::new(); + endpoints.insert("ep-json".to_string(), endpoint(Some(OutputFormat::Json))); + endpoints.insert("ep-script".to_string(), endpoint(None)); + let (app, _) = unified_api::AppBuilder::new() + .sources(sources) + .endpoints(endpoints) + .build_with_state(); + // Before the sync a call would answer 503: no numbers, and why. + let (_, body) = request(app.clone(), "GET", "/api/v1/endpoints/ep-json/counts").await; + let counts: serde_json::Value = serde_json::from_str(&body).unwrap(); + assert_eq!(counts["hosts"], serde_json::Value::Null); + assert_eq!(counts["missing_sources"], serde_json::json!(["src-test"])); + + let (status, _) = request(app.clone(), "POST", "/api/v1/sources/src-test/sync").await; + assert_eq!(status, StatusCode::OK); + + let (status, body) = request(app.clone(), "GET", "/api/v1/endpoints/ep-json/counts").await; + assert_eq!(status, StatusCode::OK, "{body}"); + let counts: serde_json::Value = serde_json::from_str(&body).unwrap(); + let (_, body) = request(app.clone(), "GET", "/api/v1/endpoints/ep-json").await; + let output: serde_json::Value = serde_json::from_str(&body).unwrap(); + let hosts = output["hostvars"].as_object().unwrap().len(); + assert!(hosts > 0, "the sample source has hosts"); + assert_eq!(counts["hosts"], hosts); + assert_eq!( + counts["groups"], + output["groups"].as_object().unwrap().len() + ); + + let (_, body) = request(app.clone(), "GET", "/api/v1/endpoints/ep-script/counts").await; + let counts: serde_json::Value = serde_json::from_str(&body).unwrap(); + assert_eq!(counts["hosts"], serde_json::Value::Null); + + let (status, _) = request(app, "GET", "/api/v1/endpoints/ep-ghost/counts").await; + assert_eq!(status, StatusCode::NOT_FOUND); +} diff --git a/tests/ui_assets_test.rs b/tests/ui_assets_test.rs new file mode 100644 index 0000000..0469451 --- /dev/null +++ b/tests/ui_assets_test.rs @@ -0,0 +1,158 @@ +use axum::body::Body; +use axum::http::{Request, StatusCode, header}; +use std::collections::HashMap; +use tower::ServiceExt; + +// The built UI served by the API itself (server.ui.assets_dir), end to end +// through the real router, against a stand-in build in a temp directory. + +const INDEX: &str = "unified-api
"; + +struct Served { + app: axum::Router, + // Held so the directory outlives the requests. + _dir: tempfile::TempDir, +} + +fn app(enabled: bool) -> Served { + let dir = tempfile::tempdir().unwrap(); + std::fs::write(dir.path().join("index.html"), INDEX).unwrap(); + std::fs::create_dir(dir.path().join("assets")).unwrap(); + std::fs::write(dir.path().join("assets/index-3f2a.js"), "console.log(1)").unwrap(); + let ui = unified_api::config::UiConfig { + enabled, + assets_dir: Some(dir.path().to_str().unwrap().to_string()), + ..Default::default() + }; + Served { + app: unified_api::AppBuilder::new() + .sources(HashMap::new()) + .ui(ui) + .build(), + _dir: dir, + } +} + +async fn get(app: &axum::Router, uri: &str, accept: &str) -> (StatusCode, String, String, String) { + let resp = app + .clone() + .oneshot( + Request::get(uri) + .header(header::ACCEPT, accept) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + let status = resp.status(); + let header_of = |name| { + resp.headers() + .get(name) + .and_then(|v: &header::HeaderValue| v.to_str().ok()) + .unwrap_or("") + .to_string() + }; + let (content_type, cache) = ( + header_of(header::CONTENT_TYPE), + header_of(header::CACHE_CONTROL), + ); + let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX) + .await + .unwrap(); + ( + status, + content_type, + cache, + String::from_utf8_lossy(&bytes).to_string(), + ) +} + +const BROWSER: &str = "text/html,application/xhtml+xml,*/*;q=0.8"; + +#[tokio::test] +async fn the_app_loads_at_the_root_and_on_every_page_of_it() { + let served = app(true); + for path in ["/", "/sources", "/sources/src-a?host=web-01", "/topology"] { + let (status, content_type, cache, body) = get(&served.app, path, BROWSER).await; + assert_eq!(status, StatusCode::OK, "{path}"); + assert!( + content_type.starts_with("text/html"), + "{path}: {content_type}" + ); + assert_eq!(body, INDEX, "{path}"); + // The page names this release's files: never kept across a release. + assert_eq!(cache, "no-cache", "{path}"); + } +} + +#[tokio::test] +async fn built_files_are_served_and_kept_for_good() { + let served = app(true); + let (status, content_type, cache, body) = + get(&served.app, "/assets/index-3f2a.js", "*/*").await; + assert_eq!(status, StatusCode::OK); + assert!(content_type.contains("javascript"), "{content_type}"); + assert!(cache.contains("immutable"), "{cache}"); + assert_eq!(body, "console.log(1)"); +} + +// An API path nothing matched must stay the API's 404: a client reading a +// 200 HTML page as an answer is worse than an error. +#[tokio::test] +async fn an_unknown_api_path_is_still_a_json_404() { + let served = app(true); + for path in ["/api/v1/no-such-route", "/api", "/api-docs/nope"] { + let (status, content_type, _, body) = get(&served.app, path, "application/json").await; + assert_eq!(status, StatusCode::NOT_FOUND, "{path}"); + assert!(content_type.starts_with("application/json"), "{path}"); + assert!(body.contains("\"error\""), "{path}: {body}"); + } +} + +#[tokio::test] +async fn the_api_answers_as_before() { + let served = app(true); + let (status, content_type, _, body) = + get(&served.app, "/api/v1/sources", "application/json").await; + assert_eq!(status, StatusCode::OK); + assert!(content_type.starts_with("application/json")); + assert!(body.starts_with('[')); + let (status, _, _, _) = get(&served.app, "/healthz", "*/*").await; + assert_eq!(status, StatusCode::OK); +} + +// /metrics: the Metrics page for a browser, the exposition for Prometheus. +#[tokio::test] +async fn metrics_is_a_page_for_a_browser_and_text_for_a_scraper() { + let served = app(true); + let (_, content_type, _, body) = get(&served.app, "/metrics", BROWSER).await; + assert!(content_type.starts_with("text/html"), "{content_type}"); + assert_eq!(body, INDEX); + let (status, content_type, _, body) = get(&served.app, "/metrics", "text/plain").await; + assert_eq!(status, StatusCode::OK); + assert!(content_type.starts_with("text/plain"), "{content_type}"); + assert!(!body.contains("