feat(sdk-metrics): align PeriodicMetricReader export timeout semantics - #8684
feat(sdk-metrics): align PeriodicMetricReader export timeout semantics#8684Rajkaran-122 wants to merge 10 commits into
Conversation
Pull request dashboard statusWaiting on the author · refreshed 2026-08-22 14:08 UTC Respond to 4 review items (e.g. link a commit, explain why not, ask a follow-up): Status above doesn't look right?
|
…cheduler Replace scheduler-based withTimeout() with CompletableResultCode.join(), matching the established pattern in BatchSpanProcessor and BatchLogRecordProcessor. The previous approach used scheduler.schedule() which throws RejectedExecutionException during shutdown because the scheduler is intentionally shut down before the final export flush.
|
Hi @Rajkaran-122 — just a friendly reminder that this pull request is waiting on you. The dashboard status comment has the open items and is kept current.
|
| * Sets the timeout for the underlying exporter. If unset, defaults to {@value | ||
| * DEFAULT_EXPORT_TIMEOUT_MILLIS}ms. | ||
| * | ||
| * @since 1.40.0 |
There was a problem hiding this comment.
Remove the since annotations. They're wrong and are added as part of the release process anyway
| * | ||
| * @since 1.40.0 | ||
| */ | ||
| public PeriodicMetricReaderBuilder setExporterTimeout(long timeout, TimeUnit unit) { |
There was a problem hiding this comment.
This is new public API surface area, which comes with new content checks into /docs/apidiffs. Please run the build to generate this.
Also, you'll see failing tests if you run the build. You'll want to fix those failing tests, and add new tests for this specific feature.
| if (maxExportBatchSize == 0) { | ||
| return exporter.export(metricData); | ||
| CompletableResultCode result = exporter.export(metricData); | ||
| result.join(exporterTimeoutNanos, TimeUnit.NANOSECONDS); |
There was a problem hiding this comment.
This change in semantic (here and below) causes callers to be blocked waiting for the timeout. I don't think this is a desirable or necessary to add a timeout.
| public final class PeriodicMetricReaderBuilder { | ||
|
|
||
| static final long DEFAULT_SCHEDULE_DELAY_MINUTES = 1; | ||
| static final int DEFAULT_EXPORT_TIMEOUT_MILLIS = 30_000; |
There was a problem hiding this comment.
Adding a default timeout when none exists today is problematic, since it can change behavior that has been otherwise working. We either have to call the lack of a timeout a bug, or be more conservative and set the default timeout to the interval when the user doesn't explicitly set it.
There was a problem hiding this comment.
Thanks @jack-berg sir or the feedback. I’ve addressed the API review comments, generated the API diff, and added tests for the new timeout API.
I also updated the timeout implementation to avoid blocking the PeriodicMetricReader scheduler while preserving asynchronous batching and handling shutdown/final export safely.
The full CI matrix is now passing. I’d appreciate your review of the updated implementation.
There was a problem hiding this comment.
This comment is not addressed. To address, you would need to leave exporterTimeoutNanos as null, and resolve the value conditionally in build().
…ication - Added setExporterTimeout() API to PeriodicMetricReaderBuilder with 30-second default per spec - Removed @SInCE 1.40.0 annotations as requested by reviewer - Updated API-diff to reflect new public API - Added comprehensive tests for timeout behavior - Implemented conditional timeout enforcement (only when explicitly configured) - Addressed blocking concern by making timeout opt-in via setExporterTimeout()
|
…Reader Add asynchronous timeout enforcement for MetricExporter operations in PeriodicMetricReader using a dedicated timeout executor. This preserves the existing asynchronous scheduling and batching behavior while enforcing the spec-required 30-second default export timeout. Changes: - Add dedicated ScheduledExecutorService for timeout scheduling - Implement applyTimeout() method with asynchronous timeout enforcement - Preserve async batch processing with Iterator-based sequential execution - Fix Error Prone warnings (UnusedVariable, PreferJavaTimeOverload) - Add timeout enforcement test The timeout executor is shut down after final export completes to avoid RejectedExecutionException during shutdown. Timeout enforcement fails the result when timeout expires without blocking the periodic scheduler. Resolves CI compilation failures in PR open-telemetry#8684.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8684 +/- ##
============================================
- Coverage 91.48% 91.29% -0.20%
- Complexity 10467 10479 +12
============================================
Files 1021 1006 -15
Lines 27694 28310 +616
Branches 3247 3575 +328
============================================
+ Hits 25337 25846 +509
- Misses 1615 1674 +59
- Partials 742 790 +48 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| } | ||
| } | ||
|
|
||
| private static class SlowMetricExporter implements MetricExporter { |
There was a problem hiding this comment.
Combine this with FastMetricExporter and just set the sleep time to 0 for the fast case.
| } | ||
|
|
||
| @Test | ||
| void explicitTimeout_exporterCompletesBeforeTimeout() throws Exception { |
There was a problem hiding this comment.
This and the test below is (and maybe all the validation cases could be parameterized tests
| private CompletableResultCode applyTimeout(CompletableResultCode result) { | ||
| if (exporterTimeoutNanos == Long.MAX_VALUE) { | ||
| return result; | ||
| } | ||
| CompletableResultCode timeoutResult = new CompletableResultCode(); | ||
| AtomicBoolean timedOut = new AtomicBoolean(false); | ||
| ScheduledFuture<?> timeoutFuture = | ||
| timeoutExecutor.schedule( | ||
| () -> { | ||
| if (!result.isDone()) { | ||
| timedOut.set(true); | ||
| logger.log( | ||
| Level.WARNING, "Export timed out after " + exporterTimeoutNanos + "ns"); | ||
| timeoutResult.fail(); | ||
| } | ||
| }, | ||
| exporterTimeoutNanos, | ||
| TimeUnit.NANOSECONDS); | ||
| result.whenComplete( | ||
| () -> { | ||
| if (timeoutFuture != null) { | ||
| timeoutFuture.cancel(false); | ||
| } | ||
| if (!timedOut.get()) { | ||
| if (result.isSuccess()) { | ||
| timeoutResult.succeed(); | ||
| } else { | ||
| timeoutResult.fail(); | ||
| } | ||
| } | ||
| }); | ||
| return timeoutResult; | ||
| } |
There was a problem hiding this comment.
| private CompletableResultCode applyTimeout(CompletableResultCode result) { | |
| if (exporterTimeoutNanos == Long.MAX_VALUE) { | |
| return result; | |
| } | |
| CompletableResultCode timeoutResult = new CompletableResultCode(); | |
| AtomicBoolean timedOut = new AtomicBoolean(false); | |
| ScheduledFuture<?> timeoutFuture = | |
| timeoutExecutor.schedule( | |
| () -> { | |
| if (!result.isDone()) { | |
| timedOut.set(true); | |
| logger.log( | |
| Level.WARNING, "Export timed out after " + exporterTimeoutNanos + "ns"); | |
| timeoutResult.fail(); | |
| } | |
| }, | |
| exporterTimeoutNanos, | |
| TimeUnit.NANOSECONDS); | |
| result.whenComplete( | |
| () -> { | |
| if (timeoutFuture != null) { | |
| timeoutFuture.cancel(false); | |
| } | |
| if (!timedOut.get()) { | |
| if (result.isSuccess()) { | |
| timeoutResult.succeed(); | |
| } else { | |
| timeoutResult.fail(); | |
| } | |
| } | |
| }); | |
| return timeoutResult; | |
| } | |
| private CompletableResultCode applyTimeout(CompletableResultCode result) { | |
| if (exporterTimeoutNanos == Long.MAX_VALUE) { | |
| return result; | |
| } | |
| CompletableResultCode timeoutResult = new CompletableResultCode(); | |
| ScheduledFuture<?> timeoutFuture = | |
| scheduler.schedule( | |
| () -> { | |
| logger.log(Level.WARNING, "Export timed out after " + exporterTimeoutNanos + "ns"); | |
| timeoutResult.fail(); | |
| }, | |
| exporterTimeoutNanos, | |
| TimeUnit.NANOSECONDS); | |
| result.whenComplete( | |
| () -> { | |
| timeoutFuture.cancel(false); | |
| if (result.isSuccess()) { | |
| timeoutResult.succeed(); | |
| } else { | |
| timeoutResult.fail(); | |
| } | |
| }); | |
| return timeoutResult; | |
| } |
| this.intervalNanos = intervalNanos; | ||
| this.exporterTimeoutNanos = exporterTimeoutNanos; | ||
| this.scheduler = scheduler; | ||
| this.timeoutExecutor = |
There was a problem hiding this comment.
Instead of a new schedule executor and the logic to shut it down, can expand the existing scheduler initialized in the builder to have 2 threads.
… jack-berg review - Remove separate timeout executor and reuse existing scheduler - Increase scheduler from 1 to 2 threads to handle both periodic exports and timeout tasks - Simplify applyTimeout() by removing AtomicBoolean/timedOut state tracking - Combine SlowMetricExporter and FastMetricExporter into single DelayingMetricExporter - Add RejectedExecutionException handling for shutdown race condition - Fix BooleanParameter warnings in tests
|
Thanks, @jack-berg sir. I've updated the implementation based on your feedback:
I also handled the scheduler-shutdown case in applyTimeout() because the final shutdown export can race with scheduler shutdown when scheduling the timeout task. All relevant sdk:metrics formatting, compilation, and PeriodicMetricReaderTest checks pass. |
|
@jack-berg sir , please review this pr. |
|
I ran JAIPilot Cloud against this exact PR head. It found one deterministic follow-up: when an export is already complete, skip creating and immediately cancelling the timeout task. The focused path changed from 1 schedule/cancel pair to 0. Baseline passed 31 focused tests, candidate passed 32 including the new regression test, and both clean Cloud-generated draft and evidence: skrcode#2 Feel free to merge or cherry-pick if it fits the intended timeout semantics. |
|
@jack-berg sir, please review the pr. |
Fixes #8311
Summary
Align
PeriodicMetricReaderexport timeout behavior with the Metrics specification by enforcing an exporter timeout for each export batch.Changes
PeriodicMetricReaderBuilder30sPeriodicMetricReaderTesting
Notes
This change updates export timeout behavior without affecting metric collection or scheduling semantics.