Skip to content

lease: reopen the keepAlive stream when the server ends it - #1628

Open
q-llewellynthomas wants to merge 1 commit into
etcd-io:mainfrom
q-llewellynthomas:fix/lease-keepalive-stream-end
Open

q-llewellynthomas wants to merge 1 commit into
etcd-io:mainfrom
q-llewellynthomas:fix/lease-keepalive-stream-end

Conversation

@q-llewellynthomas

Copy link
Copy Markdown

LeaseImpl opened the shared LeaseKeepAlive stream with no end handler, so when the server ended the stream with status OK (e.g. an etcd member shutting down gracefully, or a proxy closing the stream) the stream was dead but the KeepAlive service still considered itself running. Every lease, including ones added later, stopped being renewed and expired at its TTL, and nothing restarted the stream until the client was rebuilt.

The stream is now reopened after a clean end, on the same 500ms schedule used after an error. Observers are not notified, since the leases are still valid. keepAliveOnce also completes exceptionally on a clean end with no response, rather than leaving its future incomplete forever.

Fixes #1383

LeaseImpl opened the shared LeaseKeepAlive stream with no end handler, so
when the server ended the stream with status OK (e.g. an etcd member
shutting down gracefully, or a proxy closing the stream) the stream was
dead but the KeepAlive service still considered itself running. Every
lease, including ones added later, stopped being renewed and expired at
its TTL, and nothing restarted the stream until the client was rebuilt.

The stream is now reopened after a clean end, on the same 500ms schedule
used after an error. Observers are not notified, since the leases are
still valid. keepAliveOnce also completes exceptionally on a clean end
with no response, rather than leaving its future incomplete forever.

Fixes etcd-io#1383

Signed-off-by: Llewellyn Thomas <llewellynthomas@quantexa.com>
@kubernetes-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: q-llewellynthomas
Once this PR has been reviewed and has the lgtm label, please assign vorburger for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

leaseClient.keepAlive several minutes later will stop

1 participant