Add support for custom interruption detectors

[psytrxops]

Feature Type

I cannot use LiveKit without it

Feature Description

LiveKit Agents currently supports two interruption detection strategies through InterruptionOptions.mode:

"vad" — VAD-based interruption detection
"adaptive" — LiveKit's adaptive interruption detector

I would like to propose allowing applications to provide a custom interruption detector, in the same way that AgentSession already allows applications to provide a custom turn detector through turn_detection.

For example:

AgentSession(
    turn_handling={
        "interruption": {
            "mode": MyInterruptionDetector(),
        },
    },
)

The motivation is that interruption detection and turn detection are increasingly application-specific. In particular, production voice agents may want to combine:

VAD
+
streaming STT partial transcripts
+
semantic analysis
+
conversation context

to distinguish genuine barge-ins from backchannels and conversational acknowledgements.

For example:

Agent: "The subscription costs $49 per month and includes..."

User: "yeah"

should generally not interrupt the agent.

Whereas:

Agent: "The subscription costs $49 per month and includes..."

User: "yeah but I actually wanted..."

should interrupt the agent.

A VAD-only detector cannot make this distinction, while a semantic interruption detector can use the streaming STT transcript to make a more informed decision.

Proposed API

I suggest introducing a public interruption-detector protocol/interface, conceptually similar to the existing _TurnDetector / _StreamingTurnDetector abstractions.

For example:

class InterruptionDetector(Protocol):
    async def predict_interruption(
        self,
        *,
        ...
    ) -> bool:
        ...

For streaming detectors, an interface could expose lifecycle methods analogous to the streaming turn detector:

class StreamingInterruptionDetector(Protocol):
    def stream(
        self,
        *,
        conn_options: APIConnectOptions = DEFAULT_API_CONNECT_OPTIONS,
    ) -> InterruptionDetectorStream:

The exact interface can of course be adapted to LiveKit's existing interruption pipeline.

The important part is that AgentSession should accept a detector implementation supplied by the application rather than requiring the detector to be one of LiveKit's hard-coded modes.

Conceptually:

InterruptionDetectionMode = (
Literal["vad", "adaptive"]
| InterruptionDetector
| StreamingInterruptionDetector
)

This would make interruption detection symmetrical with the existing turn-detection API, where TurnDetectionMode already accepts custom _TurnDetector and _StreamingTurnDetector implementations.

Why this is useful

This would enable several important use cases without requiring users to fork livekit-agents:

Self-hosted interruption models

Organizations could run their own interruption classifier locally instead of depending on LiveKit's inference service.

Domain-specific interruption detection

A call-center application could train a detector specifically for its vocabulary and conversational patterns.

Semantic interruption detection

Applications could combine VAD with streaming STT:

Audio
│
▼
VAD
│
▼
Streaming STT
│
▼
Semantic interruption detector
│
├── BACKCHANNEL → continue agent speech
│
└── INTERRUPT → interrupt SpeechHandle

Different interruption policies

A customer-support agent, sales agent, IVR system, and conversational assistant may all want different definitions of an interruption.

No LiveKit fork required

Applications should be able to implement this behavior through the public AgentSession API instead of modifying internal agent_activity.py / audio_recognition.py logic.

Relationship with VAD and STT

I do not propose replacing VAD.

VAD should remain responsible for detecting acoustic speech activity, while the custom interruption detector determines whether that speech should actually cause the currently playing agent speech to be interrupted.

This distinction is important because VAD, STT, turn detection, and interruption detection represent different signals:

VAD
"Is the user speaking?"

STT
"What is the user saying?"

Turn detection
"Has the user finished their turn?"

Interruption detection
"Does the user want the agent to yield?"

A custom detector should therefore be able to consume the same streaming signals already available inside AgentSession.

Backward compatibility

Existing behavior should remain unchanged.

For example:

turn_handling={
    "interruption": {
        "mode": "adaptive",
    }
}

would continue using LiveKit's adaptive interruption detector.

Likewise:

turn_handling={
    "interruption": {
        "mode": "vad",
    }
}

would continue using VAD.

The new API would simply add a third option:

turn_handling={
    "interruption": {
        "mode": MyInterruptionDetector(),
    }
}

or an equivalent API consistent with the existing TurnDetectionMode design.

Implementation considerations

The custom detector should be integrated at the same abstraction level as the existing adaptive interruption detector, so that the rest of the interruption lifecycle remains owned by LiveKit.

The custom detector should only be responsible for producing the interruption decision.

This would keep the feature small while making the interruption subsystem extensible.

There is also an existing distinction in the code between VAD-driven speech activity and STT-driven turn detection. Providing a pluggable interruption detector would allow applications to combine these signals without coupling the interruption policy to either one.

Workarounds / Alternatives

Currently, applications that need semantic interruption detection have to work around the internal interruption pipeline or maintain a fork of livekit-agents. By modifying agent_session and agent_activity, we could pass our custom InterruptionDetector using the same underlying channels and events as provided.

A possible workaround is to disable LiveKit's interruption handling and implement interruption logic externally using VAD/STT events. However, this means reimplementing or coupling to internal agent behavior in order to correctly interrupt the active SpeechHandle and preserve the expected interruption lifecycle.

Another workaround is to use min_duration / min_words and the adaptive detector configuration, but these parameters cannot express application-specific semantic rules such as distinguishing:

"yeah"
"okay"
"right"
"hmm"

from:

"wait"
"stop"
"actually..."
"no, I meant..."
"let me explain..."

A public detector interface would avoid these workarounds while preserving the existing default behavior.

Additional Context

This request is related to the existing discussion around making interruption handling more extensible and decoupled from VAD.

In particular, LiveKit issue #6033 requests a pluggable interruption detector interface for self-hosted/custom interruption models.

There are also existing issues around the coupling between VAD, STT, and interruption behavior, including #5580 and #3427.

The proposed API follows an abstraction that already exists for turn detection: TurnDetectionMode can accept a custom _TurnDetector or _StreamingTurnDetector. Extending the interruption subsystem with an analogous abstraction would make the two systems much more consistent.

The goal is not to change the default interruption behavior, but to expose the existing interruption decision point as a supported extension point.