Skip to content

Log failed lease releases when a Shard stops #3705 - #3733

Merged
He-Pin merged 1 commit into
apache:mainfrom
pjfanning:fix-3705-shard-lease-release
Oct 30, 2026
Merged

He-Pin merged 1 commit into
apache:mainfrom
pjfanning:fix-3705-shard-lease-release

Conversation

@pjfanning

Copy link
Copy Markdown
Member

Motivation

Shard released its lease in postStop and discarded the returned Future, so a release that failed or returned false was never logged. The shard's next allocation then stalls until the lease times out, and the logs don't say why. This is the same problem as #3685, which #3699 fixes for ClusterSingletonManager.

Modification

  • Shard.postStop now calls a releaseLease() helper. It logs an error from the future's callback when the release fails or returns false. The ActorLogging adapter is thread-safe, so it is fine to use after the actor stops.
  • A release() that throws synchronously is captured with Try, so the rest of postStop still runs.
  • Added TestLease.setNextReleaseResult. It is identical to the version in Release and report singleton leases on manager termination #3685 #3699, so whichever PR merges second only needs a trivial merge.

Result

When a shard stops and its lease is not released, an error naming the type and shard id is logged.

Tests

  • sbt "cluster-sharding/testOnly org.apache.pekko.cluster.sharding.ShardWithLeaseSpec": 8 passed. The 3 new cases (release returns false, release future fails, release throws) fail without the Shard change.
  • scalafmt run on the changed files.

References

Fixes #3705

Motivation:
Shard released its lease in postStop and discarded the returned Future,
so a release that failed or returned false was never logged. The next
allocation of that shard then stalls until the lease times out with no
explanation in the logs.

Modification:
Release the lease through a helper that logs an error from the future's
callback when the release fails or returns false. A release() that
throws synchronously is captured with Try, so the rest of postStop still
runs. Add TestLease.setNextReleaseResult (same as in apache#3699) so tests can
control the release outcome.

Result:
Failed lease releases on shard stop are logged with the type name and
shard id.

Tests:
- sbt "cluster-sharding/testOnly org.apache.pekko.cluster.sharding.ShardWithLeaseSpec" - 8 passed; the 3 new cases fail without the Shard change

References:
Fixes apache#3705
@pjfanning
pjfanning requested a review from He-Pin October 29, 2026 20:36
@pjfanning pjfanning added this to the 2.0.0-M5 milestone Oct 29, 2026

@He-Pin He-Pin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Shard mixes in plain ActorLogging, whose log caches a BusLogging adapter, and it is already initialised during field construction (new Entities(log, ...)), so capturing it in a local before the actor finishes is safe to use from the future callback. Future.fromTry(Try(l.release())).flatten correctly covers both the synchronous-throw and the failed-future cases, and import context.dispatcher is at class scope so it resolves. The tn/id locals are redundant (constructor vals are stable) but harmless. Wording matches the existing ClusterSingletonManager precedent plus Shard's "{}: " typeName convention. TestLease.release() keeps its old default and resets after one use, so other specs are unaffected. Expect a trivial TestLease conflict with #3699 for whichever merges second, as noted.

@He-Pin
He-Pin merged commit 229ba06 into apache:main Oct 30, 2026
10 checks passed
pjfanning added a commit to pjfanning/incubator-pekko that referenced this pull request Oct 30, 2026
Add apache#3727, apache#3728, apache#3731, apache#3734, apache#3738 (TLS), apache#3730 (fanout publisher
performance) and apache#3733 (shard lease release logging).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pjfanning added a commit to pjfanning/incubator-pekko that referenced this pull request Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Shard: a failed lease release in postStop is silently ignored

2 participants