Skip to content

[P2] Keep FastStream observability alive through complete shutdown #434

Closed

Description

@lesnik512

Priority: P2. Recommended before 2.0. Audit rank: 3.

Audited commit: 55903b3ccc19c77cb57b74332ee67bc2e61fedcd (2026-11-04). Reproductions ran on Python 3.14.7. Relevant installed versions: FastAPI 0.162.0, Starlette 1.10.0, FastStream 0.7.8, FastMCP 4.1.5, Sentry 2.73.0, structlog 26.1.0, prometheus-client 0.26.0 and OTel SDK 1.45.2. Floor validation remains part of implementation.

Problem and evidence

FastStreamBootstrapper attaches
teardown through on_shutdown. Installed FastStream 0.7.8 runs these hooks before broker.stop()
(faststream/_internal/application.py:284) and exits the user lifespan after shutdown
(faststream/asgi/app.py:249). An actual ASGI lifespan with only broker network start/stop mocked
exports a during-service span, but drops the user-shutdown span with the SDK message
Processor is already shutdown, ignoring call.

A preceding user shutdown callback that raises can also prevent the teardown hook from running;
this follows the sequential hook loop and is the same lifecycle ownership problem.

Bracket the complete lifecycle, with teardown in finally after broker/user cleanup. Merely
moving to after_shutdown still runs before user lifespan exit. Validate supported FastStream
floor/current versions, startup failure, shutdown failure and argument preservation before
choosing the hook. Record the ordering change in the migration guide under repository rules.

Reproduction

Save each block to a temporary .py file and run it in a fresh process from the repository using uv run --no-sync python /path/to/repro.py with the project development environment installed. Telemetry uses synthetic values and in-memory/mocked transports.

FastStream shutdown span

import contextlib
from unittest.mock import AsyncMock, patch
from faststream.asgi import AsgiFastStream
from faststream.redis import RedisBroker
from starlette.testclient import TestClient
from lite_bootstrap import FastStreamBootstrapper, FastStreamConfig
from opentelemetry.trace import get_tracer_provider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter


@contextlib.asynccontextmanager
async def lifespan():
    yield
    with get_tracer_provider().get_tracer("audit").start_as_current_span("user-shutdown"):
        pass


broker = RedisBroker()
app = AsgiFastStream(broker, lifespan=lifespan)
bootstrapper = FastStreamBootstrapper(
    FastStreamConfig(
        application=app,
        logging_enabled=False,
        opentelemetry_log_traces=True,
    )
)
bootstrapper.bootstrap()
exporter = InMemorySpanExporter()
get_tracer_provider().add_span_processor(SimpleSpanProcessor(exporter))
with patch.object(broker, "start", new=AsyncMock()), patch.object(broker, "stop", new=AsyncMock()), TestClient(app):
    with get_tracer_provider().get_tracer("audit").start_as_current_span("during-service"):
        pass
print("exported", [span.name for span in exporter.get_finished_spans()])
print("bootstrapper still active", bootstrapper.is_bootstrapped)

Acceptance criteria

  • Start with a failing lifespan test proving a user-shutdown span is exported.
  • Keep telemetry active through broker drain and user lifespan shutdown, then teardown exactly once.
  • Cleanup still runs when startup or a shutdown callback fails; preserve the original error and lifespan arguments.
  • Validate the lifecycle hook against supported FastStream floor/current versions and update the migration guide.

Activity

  1. added this to the 2.0.0 milestone on Nov 4, 2026
  2. added
    bugSomething isn't working
    ready-for-agentFully specified, ready for an AFK agent
    on Nov 4, 2026
  3. lesnik512 commented on Nov 4, 2026

    @lesnik512
    MemberAuthor

    Lifecycle contract spike completed in throwaway branch prototype/lifecycle-contract, commit 2321a3e. This branch is evidence, not a merge candidate. Production code and PR #443 are unchanged.

    Report, runnable examples, and exact simplification list · recorded observations

    Compared framework-lifespan ownership and an explicit context around the runner using the same telemetry owner. 34 isolated runs: 33 passed; one deliberately demonstrates that direct FastStream start/stop never enters a lifespan. Both designs preserve shutdown spans, primary exceptions, cleanup-failure notes, and externally owned tracer providers. Two FastStream 0.6.0 spot checks passed. This is tracing-only evidence, not complete integration or worker/reload validation.

    Recommendation: keep framework lifespans as the normal owner; require explicit cleanup for manually driven operation. Keep the existing bootstrapper interface for 2.0 and share ownership/error policy internally. This would remove #443's stop-method replacement and active-lifespan flag. It is an intentional compatibility change that needs to be agreed and written into this issue before revising #443. If automatic manual cleanup remains required, retain the fallback.

    No issue contract, PR implementation, or release decision has been changed by this spike.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingready-for-agentFully specified, ready for an AFK agent

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions