Skip to content

Operational Resilience: Error Codes, Retention, Concurrency, and Dead-Letter Recovery - #325

Merged
BigDella merged 1 commit into
Parcel-Protocol:mainfrom
girly-coder01:feat/operational-resilience-and-error-handling
Oct 21, 2026
Merged

BigDella merged 1 commit into
Parcel-Protocol:mainfrom
girly-coder01:feat/operational-resilience-and-error-handling

Conversation

@girly-coder01

Copy link
Copy Markdown

Summary

Implement comprehensive operational resilience improvements across four critical areas:

Issue #318: Stable API Error Codes

  • Added 6 new error codes with HTTP status and remediation guidance
  • Covers concurrency (DUPLICATE_SUBMISSION, STALE_VERSION) and operational scenarios (DEAD_LETTER_QUEUE_FULL, JOB_SCHEDULING_ERROR, etc.)
  • All codes mapped to user-facing rejection explanations with actionable recovery steps

Issue #319: Retention and Deletion Tests

  • 80+ test cases covering retention policies, TTL enforcement, and deletion edge cases
  • Authorization paths with correlation ID preservation
  • Recovery and rollback behavior validation
  • Tests for normal flows and failure modes

Issue #320: Concurrency Tests

  • 70+ test cases for duplicate submission detection and conflict resolution
  • Stale version handling and optimistic concurrency control
  • High-contention race condition scenarios
  • Cross-user authorization enforcement
  • Idempotent deduplication and replay semantics

Issue #317: Dead-Letter Recovery

  • 60+ test cases for resumable jobs with state persistence
  • Dead-letter queue handling with failure context preservation
  • Job recovery procedures with idempotent retry semantics
  • Exponential backoff and transient failure handling
  • Cancellation and concurrent recovery safety

Test Plan

  • Retention and deletion edge cases covered
  • Concurrent duplicate submission scenarios tested
  • Stale version handling validated
  • Dead-letter queue recovery procedures tested
  • Error codes and remediation guidance implemented
  • Authorization paths for all operations verified

Breaking Changes

None. All changes are additive and maintain backward compatibility.

Closes #317
Closes #318
Closes #319
Closes #320

…oncurrency tests, and dead-letter recovery

Implement comprehensive improvements for operational reliability and error handling:

1. Add stable API error codes and remediation guidance (Closes Parcel-Protocol#318)
   - New error codes for concurrency scenarios: DUPLICATE_SUBMISSION, STALE_VERSION
   - New error codes for operational scenarios: DEAD_LETTER_QUEUE_FULL, JOB_SCHEDULING_ERROR, METADATA_RETENTION_EXPIRED, OPERATION_RECOVERY_FAILED
   - Update rejection reasons with appropriate categories and recovery guidance
   - Map all error codes to user-facing rejection explanations

2. Add retention and deletion tests for operational metadata (Closes Parcel-Protocol#319)
   - Tests for metadata retention policies and TTL enforcement
   - Deletion edge cases: non-existent operations, in-flight operations, terminal states
   - Authorization paths for deletion with correlation ID preservation
   - Recovery and rollback behavior with operation state preservation
   - Normal deletion flow and failure mode handling

3. Implement concurrency tests for duplicate submissions and stale versions (Closes Parcel-Protocol#320)
   - Duplicate submission detection with idempotency key tracking
   - Payload conflict detection for conflicting duplicate submissions
   - Stale version handling and optimistic concurrency control
   - Race condition tests for near-simultaneous and high-contention scenarios
   - Deduplication and replay semantics across retries
   - Authorization paths for deduplication with cross-user prevention

4. Implement resumable background jobs with dead-letter recovery (Closes Parcel-Protocol#317)
   - Job resumption with incremented attempt counters and state persistence
   - Idempotency key replay semantics through resumption
   - Correlation ID maintenance across job lifecycle
   - Dead-letter queue handling with failure context preservation
   - Dead-letter job inspection and recovery procedures
   - Exponential backoff and transient failure retry handling
   - Cancellation and concurrent recovery safety

All implementations follow existing code patterns and include comprehensive test coverage.
@drips-wave

drips-wave Bot commented Oct 21, 2026

Copy link
Copy Markdown

@girly-coder01 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@BigDella
BigDella merged commit 50fd1fa into Parcel-Protocol:main Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants