cli/command: use longer init-ping timeout for ssh connections - #7337
adisivaprasad wants to merge 1 commit into
Conversation
The CLI performs an initial ping to the daemon during (lazy)
initialization, which is used for API-version negotiation and to
collect server info (e.g. whether experimental features are enabled,
and the daemon's OS type). This ping has a short (2 second) timeout
to prevent the CLI from hanging when connecting to a local daemon
that is not running.
Establishing a connection through the ssh connection helper requires
a TCP connection, host key verification, and authentication before
the connection to the daemon is ready to be used, which can take
longer than this timeout, in particular when connecting to a host
for the first time. In that case, the initial ping is aborted and
API-version negotiation is skipped: the client uses its default
(maximum) API version, and requests to a daemon with a lower API
version fail with errors such as:
Error response from daemon: client version 1.56 is too new.
Maximum supported API version is 1.55
This can be observed with 'docker --debug --context <ssh-context>
version', which dials the daemon twice on a slow (first) connection,
and prints the client's default (maximum) API version instead of the
negotiated version.
The ssh connection helper sets a default ConnectTimeout=30 for the
ssh process, so allow the initial ping for ssh connections to complete
within that time, plus some additional time for the handshake and the
ping itself. Non-ssh connections (local sockets, tcp) keep the default
(short) timeout, as those connections are either established quickly,
or fail quickly when the daemon is not running.
Fixes docker#6125
Signed-off-by: adisivaprasad <adisivaprasad@users.noreply.github.com>
|
Another case hit by the 2s init-ping deadline, which this PR helps with but does not fully fix: interactive SSH authentication (passphrase-protected key loaded on demand with With such a setup, every docker command that needs to load the key prompts twice:
Reproduction without a real key (empty agent, fake askpass that waits 5s), docker 25.0.2. The relevant code is unchanged on master: About the 32s value: I'd suggest not applying an init deadline at all for
Separately, |
- What I did
Fixes #6125
The CLI performs an initial ping to the daemon during (lazy) initialization, which is used for API-version negotiation and to collect server info (e.g. whether experimental features are enabled, and the daemon's OS type). This ping has a short (2 second) timeout to prevent the CLI from hanging when connecting to a local daemon that is not running (#3652, #4226).
Establishing a connection through the ssh connection helper requires a TCP connection, host key verification, and authentication before the connection to the daemon is ready to be used, which can take longer than this timeout, in particular when connecting to a host for the first time. In that case the initial ping is aborted, and API-version negotiation is skipped: the client keeps its default (maximum) API version, and requests to a daemon with a lower API version fail with errors such as:
This can be observed with
docker --debug --context <ssh-context> version, which dials the daemon twice on a slow (first) connection (the first connection is killed when the ping's 2-second deadline expires), and prints the client's default (maximum) API version instead of the negotiated version:- How I did it
The ssh connection helper sets a default
ConnectTimeout=30for the ssh process (cli/connhelper), so this change allows the initial ping forssh://endpoints to complete within that time, plus some additional time for the handshake and the ping itself (sshInitTimeout = 32s). Non-ssh connections (local sockets, tcp) keep the default (short) timeout, as those connections are either established quickly, or fail quickly when the daemon is not running, so their fast-fail behavior is unchanged.- How I verified it
Reproduced the issue by simulating a slow ssh connection (an
sshwrapper that delays the connection by ~3 seconds, i.e. longer thandefaultInitTimeout) against a local daemon with a lower API version than the client:Before (client built from master):
After:
The initial ping now completes within the longer timeout, the connection is reused for the subsequent request (only one ssh connection is started), and the API version is negotiated correctly.
Also added unit tests:
TestGetInitTimeout: verifies the timeout selection per endpoint type (default for unix/tcp, longer for ssh, explicitinitTimeoutoverride takes precedence for both).TestInitializeFromClientSlowConnection: simulates a daemon that is slow to connect to (dial takes longer thandefaultInitTimeout), and verifies that for anssh://endpoint the initial ping completes and the API version is negotiated, whereas non-ssh endpoints keep the default fast-fail behavior.go test ./cli/command/...passes, andgolangci-lint(v2.10.1, as used in CI) reports no issues.