Repository navigation
Implement airflow triggerer #200
Description
Description / Context
A running triggerer process allows deferrable operators to be used in DAGs so that worker management can be improved e.g. on scheduler re-starts and for providing HA: see here for more details.
Problem
Currently the airflow-operator does not implement an airflow triggerer process/role.
Proposal
Introduce a new optional role to go alongside the existing ones:
- webservers
- schedulers
- celeryExecutors / kubernetesExecutors
- triggerers
This new role will encapsulate a command that starts the triggerer.
The CRD would look something like this:
---
apiVersion: airflow.stackable.tech/v1alpha1
kind: AirflowCluster
metadata:
name: airflow
spec:
...
webservers:
...
celeryExecutors:
...
schedulers:
...
triggerers:
roleGroups:
default:
replicas: 1
Acceptance criteria
- add an airflow role
triggerer - create a test that uses a DAG/deferrable operator that keeps running over a re-deployment/scheduler re-start
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
lfrancke commentedon Aug 28, 2025
Thanks. This is fine for me.
adwk67 commentedon Sep 8, 2025
Release note snippet
adwk67 commentedon Sep 9, 2025
Completed by #679
lfrancke commentedon Sep 9, 2025
Do we have docs for this?
adwk67 commentedon Sep 10, 2025
Docs (for dag-processors and triggerers) can be found here: