Fix Kafka container starting before the broker is ready

[cristianrgreco]

Summary

new KafkaContainer(image).start() resolved before the broker was listening, unless the caller had set a wait strategy with withWaitStrategy.

Root cause

KafkaContainer starts in two phases. beforeContainerCreated saves this.waitStrategy in originalWaitStrategy and replaces it with a placeholder Wait.forLogMessage("Waiting for script..."). containerStarted then copies the real starter script and waits a second time with this.selectWaitStrategy(client, inspectResult, this.originalWaitStrategy).

GenericContainer.selectWaitStrategy declares that third parameter as waitStrategy = this.waitStrategy. Since #1096, GenericContainer.waitStrategy defaults to undefined instead of Wait.forListeningPorts(). With no user strategy, originalWaitStrategy is undefined, the default parameter applies, and the second wait is the placeholder again, which the container's log already satisfies.

The failed Tests (kafka, 24.x, podman) job on main shows it for container 5627189f189a:

Time (UTC)	Event
16:38:27.442	Copied content to container (starter script)
16:38:27.486	Waiting for log message "Waiting for script..."...
16:38:27.514	Container is ready, so start() resolves
16:39:28.171	Kafka: authorizerStart completed for endpoint PLAINTEXT. Endpoint is now READY

start() resolved 61 seconds before the broker opened its port. The job log has 7362 Connection refused lines from the clients.

Fix

The third parameter of selectWaitStrategy could not express "no strategy configured", which is the only thing it was added for in #1096, so it is removed. The method now always reads this.waitStrategy.

KafkaContainer.containerStarted puts the saved strategy back on this.waitStrategy and calls this.selectWaitStrategy(client, inspectResult). The second wait is the user's strategy, or what the selector picks when there is none: a health check if the container or image defines one, otherwise listening ports. It can no longer be the placeholder.

RedpandaContainer gets the same two lines. It was not affected by the bug (below), but it was the only other caller of the third parameter.

Restoring this.waitStrategy also means a second start() on the same builder saves the user's strategy again instead of the placeholder left over from the first start.

Is listening ports a deterministic readiness signal for Kafka?

Yes. With a fixed port, Kafka opens the listener socket in Acceptor.start(), which runs only after the endpoint is ready, not when the SocketServer is constructed (4.1, 3.9). Local timelines with this change:

Image	Broker log	Kafka time	Internal port 9093/tcp ready
cp-kafka:8.3.2 (KRaft)	Endpoint is now READY for PLAINTEXT, then Kafka Server started	17:23:22.221, 17:23:22.225	17:23:22.350
cp-kafka:7.9.1 (ZooKeeper)	Awaiting socket connections on 0.0.0.0:9093, then [KafkaServer id=1] started	17:23:45.375, 17:23:45.381	17:23:45.440

Other implementations

Java and Go copy the starter script the same way and then wait for a broker log line:

• Java: Transitioning from RECOVERY to RUNNING in KRaft mode (KafkaHelper.WAIT_STRATEGY) and [KafkaServer id=\d+] started in ZooKeeper mode (KafkaContainer).
• Go: Transitioning from RECOVERY to RUNNING, in the same post-start hook that copies the script (kafka.go).

I borrowed the ordering (copy the script, then wait on a signal that the broker itself produces) and did not adopt the log line as this module's default. Listening ports was the default here before #1096, the tables above show the port opens within the same few milliseconds as those log lines, and a log-based default would hang for users who lower Kafka's log level.

Redpanda

RedpandaContainer was not starting early. Its constructor sets Wait.forLogMessage("Successfully started Redpanda!"), and withWaitStrategy only accepts a strategy, so its originalWaitStrategy was never undefined through the public API. Its change here only follows the removed parameter.

Expected conflicts

#1584 changes generic-container.ts, kafka-container.ts and redpanda-container.ts, including the lines next to these in both containerStarted methods. #1597 changes kafka-container.ts. Whichever merges second will need a small rebase. This PR depends on neither.

Verification

Regression test should be listening when started in kafka-container-latest.test.ts: right after start() resolves it opens one TCP connection to the broker's port, with no polling.

The probe runs inside the container (bash -c '</dev/tcp/localhost/9093', the same check Wait.forListeningPorts() uses) because a host-side connect cannot fail on Docker Desktop: with nothing listening on the container port, a connect to the published port was accepted after 1 ms and closed after 2 ms.

Red, on main (npx vitest run packages/modules/kafka/src/kafka-container-latest.test.ts -t "should be listening when started"):

17:22:42.206Z testcontainers [DEBUG] [7a7878d44aab] Waiting for log message "Waiting for script..."...
17:22:42.208Z testcontainers [INFO] [7a7878d44aab] Container is ready
17:22:42.217Z testcontainers [DEBUG] [7a7878d44aab] Waiting for log message "Waiting for script..."...
17:22:42.218Z testcontainers [INFO] [7a7878d44aab] Container is ready
17:22:42.219Z testcontainers [DEBUG] [7a7878d44aab] Executing command "bash -c </dev/tcp/localhost/9093"...

 FAIL  packages/modules/kafka/src/kafka-container-latest.test.ts > KafkaContainer > should be listening when started
AssertionError: expected 1 to be +0 // Object.is equality
 ❯ packages/modules/kafka/src/kafka-container-latest.test.ts:26:22

 Test Files  1 failed (1)
      Tests  1 failed | 5 skipped (6)

Green, same command with the fix:

 Test Files  1 passed (1)
      Tests  1 passed | 5 skipped (6)

The original red evidence is the failed CI job linked above.

Both Kafka test files and the Redpanda test file, on Docker Desktop with retries off (npx vitest run packages/modules/kafka/src/kafka-container-latest.test.ts packages/modules/kafka/src/kafka-container-7.test.ts packages/modules/redpanda/src/redpanda-container.test.ts):

 Test Files  3 passed (3)
      Tests  23 passed (23)

Not run locally: Podman.

npm run format, npm run lint, npm run knip and npm run check-compiles pass.

Not breaking

One signature changes: the protected GenericContainer.selectWaitStrategy(client, inspectResult, waitStrategy?) loses its optional third parameter. It is not documented, and Kafka and Redpanda were its only callers in this repository.

• Subclasses that call it with two arguments, and everything that does not call it, behave as before.
• A subclass outside this repository that passes a third argument gets a TypeScript error. In JavaScript the argument is ignored and this.waitStrategy is used.
• A wait strategy set with withWaitStrategy is used exactly as before.
• Without one, KafkaContainer.start() now waits for the exposed ports to be listening, which is what it did before Use configured health checks as the default wait strategy #1096. It takes longer to resolve only because it used to resolve too early.

Labelled patch on that basis. If the removed parameter should count as breaking, this needs major instead.