Add an SMB and NFS file-share connector

[richardmhope]

Closes #145

The architectural call

The share arrives as a mount, not as a protocol client. SMB and NFS both have
mature, kernel-side implementations every operator already knows how to configure
read-only. Re-implementing either in the engine would mean a second authentication
stack, a second set of protocol bugs, and a second place a credential lives — and
one connector then covers both protocols, because at the level the engine works at
they are the same thing: a directory tree.

The consequence is where the credential lives: in the mount, configured by
whoever runs the engine, never in a lease. POST /sources/{id}/test therefore
refuses a file share and says why rather than "not implemented yet" — the API
cannot see the engine's mounts, and "connectivity OK" reported from the wrong
machine is worse than no answer.

If a deployment genuinely cannot mount, a native SMB client is a later connector
that reuses everything below it. Nothing here forecloses that.

The three ways a filesystem walk goes wrong

It escapes. Every path is resolved and checked against the resolved root
before it is opened — resolved, because that is the only form a symlink or a ..
cannot lie about. A link into a sibling root is refused too: following it would
have two fetch tasks scan the same files and double-count the coverage manifest.

It never ends. Symlinks are not followed by default; when they are, a
visited-inode set stops a cycle. A tree deeper than 64 levels is a recorded gap
rather than a stack overflow.

It blocks. Only regular files are read. A FIFO opened for reading blocks until
somebody writes to it — a hung task, which no timeout in this connector would catch,
and the kind of failure that looks like a slow share until you look.

Everything past that is the extraction pipeline the other connectors already use:
size caps, decompression bombs, parser isolation.

Smaller calls

• Scope is what an operator asked for, never what exists. Discovery does not
  enumerate the mount's top-level directories into roots; that would make the scope
  of a scan depend on what somebody created last week.
• Identity is the path within the share, not the absolute path — remounting the
  same share elsewhere must not orphan every finding on it (ADR 0006). It doubles as
  the path display key an analyst writes a path_glob suppression against.
• Sorted at every level, which is what makes a checkpoint mean "everything up to
  this path has been handed over" and lets a reclaimed task resume without skipping
  or repeating (Support incremental and resumable source scanning #143).
• Gaps are dispositioned honestly. A permission-denied or over-the-limit file is
  a failure — it was in scope and should have been readable, so reconciliation must
  not treat its absent findings as remediated — while an image or an excluded path is
  a skip. A configured root that does not exist is a scope gap, so a typo appears on
  the manifest rather than as a scan that quietly covered less than it was asked to.

SourceType.SMB → FILESHARE

No migration. The column is a plain VARCHAR (native_enum=False, no CHECK), and the
old value was never storable — validate_connection refused it — so no row can carry
it.

Two existing guards were extended rather than worked around:

• The connection-key scan (tests/test_source_connection_schema.py) now covers all
  three connectors. Its email-alias assertion became per-model, since a share has
  no email and demanding one of every future model would make the alias table
  something every connector has to satisfy rather than a note about one.
• "Post-MVP types are refused" became set(SourceType) == set(CONNECTION_MODELS) plus
  a simulated unshipped type. The refusal path is for the next connector; deleting
  the test because every type now ships would remove the guard that catches it.

Acceptance criteria, against #145

• Mount and credential requirements are documented — docs/connectors.md § File
  shares, with the compose bind and the fstab lines
• Traversal cannot escape configured roots
• Symlink, permission, size, and malformed-file outcomes are explicit
• Checkpoints do not leak content or credentials — the position is one key, and the
  conformance kit scans the public payload with real file content as the sentinel
• Findings retain stable identity after unchanged rescans

36 tests, built on a real directory tree rather than a filesystem double: symlink
resolution, permission bits, and file modes are the kernel's behaviour, and a double
would stub out exactly what is worth testing. The permission tests skip as root, with a
guard test that says so — otherwise they would pass for the wrong reason in a container.

make check green: 1783 passed.

[icebergai-review-bot]

Verdict

APPROVE

Completed bounded review across 1 immutable scope(s). 3 findings: two material security/correctness defects and one worker-availability defect in the new fileshare connector.

Scope health

Convergence: healthy. Review mode: initial.
Recommended action: CONTINUE_INCREMENTAL.

• No escalation signals.

Prior findings

Finding	Status
—	No prior finding state

New findings

Root cause: Checkpoint ordering is based on canonical resource keys rather than the actual deterministic traversal position.

• FYI · medium: Resume can silently skip files when following an in-root symlink — packages/connectors/src/iceberg_connectors/fileshare/connector.py
  Status: NEW. Attribution: new_in_scope.
  The checkpoint is a lexicographic path key, but traversal order ceases to be lexicographic when a symlink is followed. _walk descends through the symlink before later sibling entries, while _key canonicalizes paths through the symlink target. On reclaim, entry.key <= resume_after then treats those later siblings as already delivered.
  Invariant: A valid checkpoint must resume without repeating or skipping any undispatched in-scope file.
  Ownership: FileshareConnector traversal and checkpoint protocol.. Behaviour: Reclaimed fileshare fetch tasks with followed in-root symlinks..
  Evidence: With follow_symlinks=true, create root entries alias -> z/, y.txt, and z/file.txt. Traversal yields z/file.txt through alias before y.txt; checkpointing after it causes a resumed run to skip y.txt because y.txt <= z/file.txt.
  Independent assessment: Downgraded to advisory: The stated reproduction does not hold: _walk processes the sorted directory entries in reverse and yields y.txt before it descends into the queued directory/symlink target. Also, the real z/ directory is queued before alias, causing the alias target to be recognized as seen. Thus y.txt is checkpointed before z/file.txt, not silently skipped on resume.

Root cause: The configured path is trusted as a mount without enforcing a mount or allowlisted-root boundary.

• FYI · medium: Any API administrator can scan arbitrary engine-local directories — packages/connectors/src/iceberg_connectors/fileshare/connector.py
  Status: NEW. Attribution: new_in_scope.
  mount_path is only checked for absolute syntax in the API and for existence plus directory type in _resolved_mount; it is never verified to be an approved/read-only mount. This makes the connector's engine-local filesystem reachable through source configuration, rather than limiting it to an operator-provided share mount.
  Invariant: A fileshare scan may access only the read-only share mount provisioned for that source.
  Ownership: Fileshare source configuration validation and engine-side connector initialization.. Behaviour: All fileshare scans configured through the API..
  Evidence: FileshareConnection._absolute_mount accepts any absolute path, including /; FileshareConnector._resolved_mount accepts any existing directory. Creating a fileshare source with mount_path: "/" and starting a scan therefore walks engine-local files.
  Independent assessment: Downgraded to advisory: The change accepts an existing directory path, but the supplied immutable change does not independently establish that an API administrator controls this field without an external mount allowlist or container/filesystem boundary. The claimed authorization crossing is therefore unproven.

Root cause: The limit is enforced after eager enumeration and sorting.

• FOLLOW-UP ISSUE · medium: Directory entry limit does not prevent unbounded directory-list memory use — packages/connectors/src/iceberg_connectors/fileshare/connector.py
  Status: NEW. Attribution: new_in_scope.
  _walk calls sorted(os.scandir(directory), ...) before evaluating len(entries) > MAX_ENTRIES_PER_DIRECTORY. Thus all entries in an arbitrarily large directory are materialized and sorted before the intended 50,000-entry cap can take effect.
  Invariant: A directory exceeding the configured traversal bound must not require unbounded worker memory to report its gap.
  Ownership: FileshareConnector directory traversal.. Behaviour: Fileshare roots containing very large directories..
  Evidence: The cap is applied only after entries = sorted(os.scandir(directory), key=lambda item: item.name) has completed.

Fix-induced regressions

• None evidenced.

Uncertainty

• No material uncertainty recorded.

Validation

• Reviewed the supplied immutable diff and its included tests; no prior findings were supplied for recheck.

Residual risks

• None identified.