Skip to content

fix(no-ticket): stop a stalled POST from waiting forever - #556

Open
cloudsmith-iduffy wants to merge 11 commits into
masterfrom
iduffy/fix-post-read-timeout
Open

cloudsmith-iduffy wants to merge 11 commits into
masterfrom
iduffy/fix-post-read-timeout

Conversation

@cloudsmith-iduffy

Copy link
Copy Markdown
Collaborator

Description

This PR stacks on #538. #538 added a default timeout to API requests, but a POST kept only the connect timeout. The reason: the session retries a read timeout for every method, so a read timeout on a POST that the API already processed would send it again. The result is a duplicate entitlement, token or package.

So a POST to an API that takes the request body and never replies still waits forever. For example, cloudsmith entitlements create acme/repo --name ci against a server that reads the POST and stays silent:

Before:

STILL HANGING after 90s
server saw: 1 x POST /entitlements/acme/repo/

After:

Creating ci entitlement for the repo repository ... ERROR
Failed to create the entitlement! (status: 0 - Unknown Status)

Detail: ReadTimeout
HTTPConnectionPool(host='127.0.0.1', port=28766): Read timed out. (read timeout=120)

The command fails after 120 seconds, and the server sees the POST one time.

This PR makes two changes:

  • POST and PATCH requests get a 120-second read timeout. The API CDN returns a 504 after 60 seconds without a reply, so a slow but healthy request (package create, copy, move, resync) gets a response well before the CLI stops it.
  • The retry policy does not retry a POST or PATCH after a read error. urllib3 counts a read timeout and a dropped connection after the request was sent as read errors. In both cases the API can have processed the request. Connect errors still get retries for every method, because the request never left the client. GET, PUT and DELETE keep their read retries.

The dropped-connection case had the same duplicate problem. When the server reads the POST and closes the connection, the CLI sent the POST 7 times in 62 seconds before this change. Now it sends it once and fails at once.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Refactoring
  • Other (please describe)

Additional Notes

  • The read-error rule is in the shared retry policy, so it also applies to other POSTs that use create_requests_session: the OIDC exchange, the SAML refresh and 2FA exchange, and the S3 upload POST. None of these lose a working retry. A retried refresh or TOTP exchange uses a token that the first request already spent. A retried S3 upload POST sent the full Content-Length with an empty body, because the multipart stream was already read. I checked the upload case against a local server.
  • A 5xx response to a POST still gets retries, as before. Against a server that reads the POST and returns 504, the CLI sends the POST 7 times. A 504 from the CDN while the API still processes the POST can therefore create duplicates. This PR does not change that. It needs a separate decision about status retries for POST.
  • Not breaking. A POST that hung forever now fails with an error. A PATCH that timed out at 30 seconds and got retries now waits 120 seconds one time.
  • The change in RetryWithCallback.increment is two lines after the certificate check. It merges cleanly with the increment change in fix(no-ticket): stop retrying unknown hosts and show backoff retries #542.

Test: pytest cloudsmith_cli -m "not integration". The new test runs a local server that reads each request and never replies. It counts one POST and three GETs with retries=2.

🤖 Generated with Claude Code

cloudsmith-iduffy and others added 4 commits October 23, 2026 22:39
The v1 SDK calls sent no timeout, so a host that accepts the
connection and does not reply blocked the command forever.
RestClient.request now uses a 30-second timeout when the caller gives
none. A POST gets only the connection timeout, because a retry after a
read timeout can create a second entry. The v2 client now uses this
default instead of its own copy of the rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CLI default for --error-retry-max is 6, not 5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Give POST and PATCH requests a 120-second read timeout. The API CDN
returns a 504 after 60 seconds, so a healthy API replies in time.

Do not retry a POST or PATCH request after a read timeout or a dropped
connection. The API can have processed the request, so a retry can
create a second entry. Connect errors still get retries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 24, 2026 18:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Static review found no blocking issues in the focused timeout and retry changes; tests were not executed during this review.

No open issues 🎉

What changed in this PR

Builds on #538 to bound stalled API requests without replaying POST or PATCH requests after a read error.

Changes:

  • Sets a 120-second default read timeout for POST and PATCH.
  • Stops read-error retries for those methods while preserving connection-error and status-code retries.
  • Adds local-server regression tests and updates release notes.
File Description
cloudsmith_cli/​core/​tests/​test_rest.py Tests timeout defaults and POST-versus-GET read retries.
cloudsmith_cli/​core/​session.py Prevents POST and PATCH retries after read errors.
cloudsmith_cli/​core/​rest.py Applies the longer POST and PATCH read timeout.
CHANGELOG.md Documents bounded waits and revised retry behavior.

Review effort: Balanced

The API CDN returns a 504 after 60 seconds. A 30-second read timeout
stopped a slow request that was healthy, and the session then sent it
again. The default read timeout is now 90 seconds. The connection
timeout stays at 30 seconds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cloudsmith-iduffy and others added 2 commits October 28, 2026 12:01
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from iduffy/fix-v1-api-timeout to master November 2, 2026 00:21
* origin/master: (25 commits)
  fix(ENG-15484): accept --color on click versions before 8.2 (#582)
  fix(no-ticket): stop retrying unknown hosts and show backoff retries (#542)
  fix(no-ticket): stop credential helpers from overwriting config files that do not parse (#540)
  fix(no-ticket): remove the pnpm launcher when .npmrc does not exist (#539)
  fix(no-ticket): keep the user's Cargo registry credential provider (#532)
  build(deps): bump pyasn1 (#580)
  build(deps): bump the actions-all group with 4 updates (#577)
  build(deps): bump the uv-all group with 7 updates (#578)
  build(deps): bump alpine from `294b683` to `be26560` (#576)
  fix(no-ticket): keep credentials.ini sections and save missing API keys (#533)
  fix(no-ticket): set a default timeout on each API request (#538)
  fix(no-ticket): print the result count when a list is empty (#545)
  fix(no-ticket): escape the args in the terraformrc block (#543)
  fix(no-ticket): show the dependencies help for list deps (#546)
  fix(no-ticket): show a correct summary for the pnpm credential helper (#547)
  fix(no-ticket): show a clean error when the SAML IdP response is not a JSON object (#559)
  fix(no-ticket): show the API error when mcp cannot load the API schema (#561)
  fix(no-ticket): treat an empty trailing OWNER/REPO part as not given (#550)
  fix(no-ticket): print a skipped API error one time in copy, move and resync (#552)
  fix(no-ticket): save credentials.ini with mode 0600 (#553)
  ...

# Conflicts:
#	CHANGELOG.md
#	cloudsmith_cli/core/rest.py
#	cloudsmith_cli/core/tests/test_rest.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The changelog inaccurately describes upload behavior, and regression tests leave PATCH, dropped connections, and preserved retry behavior unverified.

2 open issues

Review effort: Balanced

Comment thread CHANGELOG.md Resolved
Comment thread cloudsmith_cli/core/tests/test_rest.py Resolved
Check that a dropped connection and a read timeout cause one attempt
for POST and PATCH, and retries for GET. Check that POST and PATCH
still retry connect errors and retryable statuses. Scope the changelog
to unchanged upload and download timeouts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block approval of the POST/PATCH timeout and retry changes.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

@cloudsmith-iduffy
cloudsmith-iduffy marked this pull request as ready for review November 3, 2026 11:17
@cloudsmith-iduffy
cloudsmith-iduffy requested a review from a team as a code owner November 3, 2026 11:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues for the POST/PATCH timeout and retry changes.

No open issues 🎉

Review effort: Balanced

…d-timeout

# Conflicts:
#	CHANGELOG.md

Co-authored-by: cloudsmith-iduffy <178375997+cloudsmith-iduffy@users.noreply.github.com>
auto-merge was automatically disabled November 3, 2026 16:08

Head branch was pushed to by a user without write access

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants