[DBI-903] Strawman PR: OTel based edge collector in local dev environment

[pfcoperez]

📝 This is a Strawman PR

This PR settles the base of OTel based log delivery prototyping.

It adds an edge-collector with an example of the future configuration of ClickPipes internal log delivery services.
It also adds a service logs collector that simulates the log source within the ClickHouse integrated environment.

In this example we test the path: services -> daemon set collector -> edge collector -> CH destination

Part of: DBI-903

[claude]

Claude finished @pfcoperez's task in 10m 40s —— View job

---

Conflict resolved and pushed ✅

• Identify the conflict — both sides modified Tiltfile, ancillary-docker-compose.yml, and docker-compose-dev.yml (main moved ahead considerably since this branch forked at 2ff0207)
• Resolve the three conflicting files — kept both sides: main's otel-collector metrics service, ClickHouse TLS setup and image-pin changes, plus this PR's edge-collector / otel-collector-agent additions
• Bring the rest of main's changes into the tree
• Verify the result — git diff origin/main HEAD is now exactly this PR's 5 files, +239/−0
• Commit and push to DBI-903/local-dev/edge-collector → eae45cf

What was done

The branch content now equals current main plus this PR's changes. In the three conflicting files both sides were additive and compatible, so the resolution keeps everything:

• Tiltfile — main's empty-*_IMAGE-var cleanup loop and otel-collector dc_resource, plus this PR's edge-collector / otel-collector-agent resources.
• ancillary-docker-compose.yml — main's ClickHouse TLS entrypoint/mounts, parameterized MySQL/MariaDB images, and otel-collector (metrics) service, plus this PR's edge-collector and otel-collector-agent services.
• docker-compose-dev.yml — main's PEERDB_QUEUE_FORCE_TOPIC_CREATION and OTel metrics env vars, plus this PR's x-flow-logging anchor and the logging: entries on the flow services.

One caveat

This runner's clone is shallow and git fetch/git merge are permission-restricted, so I could not record a two-parent merge commit — eae45cf is a regular commit whose tree is the merge result. Consequences:

• The conflict is resolved content-wise and the PR should now show as mergeable.
• Until main is formally merged in, the PR's "Files changed" tab will also display the main-side changes that were folded in. Clicking Update branch on the PR (or running git merge origin/main locally and pushing — it will complete cleanly now) records the merge parent and collapses the diff back to just the 5 edge-collector files.
  · Branch

[pfcoperez]

@claude there are still some conflicts to resolve, resolve them and push to the source branch (DBI-903/local-dev/edge-collector)

[claude]

Claude finished @pfcoperez's task in 8m 31s —— View job

---

Conflicts resolved and pushed ✅

• Inspect branch vs origin/main — tree was already main + this PR's 5 files (+239/−0)
• Diagnose why GitHub still reported conflicts
• Resolve the two conflicting hunks
• Verify content is unchanged (pure relocation)
• Push to DBI-903/local-dev/edge-collector → a638d06

Why conflicts remained after the last run

The content was already correct; the merge wasn't. eae45cf folded main's content in but could not record origin/main as a merge parent — this runner denies git merge, git commit-tree, and writing .git/MERGE_HEAD. So GitHub still computes the merge from the old merge base 2ff0207, and two of this PR's additions sit at exactly the same insertion anchor as an addition main made in the meantime:

File	main adds	this PR added
Tiltfile	dc_resource('otel-collector', …) right after openssh	the edge-collector / otel-collector-agent block at that same point
ancillary-docker-compose.yml	the otel-collector service immediately before volumes:	its two services at that same point

Two same-position insertions with different text conflict even when both sides are purely additive — which is exactly the case here.

The fix

Moved this PR's two blocks into regions main does not touch. Both now sit directly after dozzle:

• Tiltfile:282 — the Ancillary-LogDelivery dc_resource calls, now separated from main's otel-collector line by 4 unchanged lines (# Test services comment, blank, toxiproxy, openssh).
• ancillary-docker-compose.yml:249 — the edge-collector and otel-collector-agent services, now ~30 lines from main's nearest change (the openssh image pin).

docker-compose-dev.yml needed no change: its x-flow-logging anchor is already 5 unchanged lines away from main's PEERDB_QUEUE_FORCE_TOPIC_CREATION addition, and the rest of its additions are byte-identical to main's, which merges cleanly.

The commit is 44 insertions(+), 44 deletions(-) — a pure relocation. git diff origin/main HEAD is still exactly this PR's 5 files, +239/−0, and the moved blocks are textually verbatim (comments included).

Verification and caveats

• Content verified: git diff origin/main HEAD --stat unchanged at +239/−0 across Tiltfile, ancillary-docker-compose.yml, docker-compose-dev.yml, volumes/otel/edge-collector.yaml, volumes/otel/otel-collector-agent.yaml.
• PR workflows re-triggered on a638d06 (they show action_required, i.e. awaiting maintainer approval, as usual for bot pushes). GitHub building those pull_request refs is a good sign the merge ref now resolves, but I could not read the API's mergeable field directly — gh and outbound fetches are blocked in this runner. Please confirm on the PR page.
• Still worth one click: because the earlier fold-in has no merge parent, the "Files changed" tab still shows main's changes alongside this PR's. Pressing Update branch (or git merge origin/main locally and pushing) records the merge parent and collapses the diff back to just the 5 files. That should now complete without conflicts.

One thing to flag (not fixed)

Three files reference volumes/otel/README.md, which this PR never adds — docker-compose-dev.yml:37, ancillary-docker-compose.yml:252, and volumes/otel/otel-collector-agent.yaml:15. I deliberately left the pointers verbatim rather than strip them, since the intent looks like a README you still mean to add. Fix this →
· Branch