At its simplest, an XCom is a key-value pair identified by a key , a task_id , and a dag_id . By default, when a task returns a value, Airflow automatically serializes that value and writes it into the metadata database (e.g., PostgreSQL or MySQL) in the xcom table.
XComArgs are reference objects that point to an XCom value that may not have been created yet. They are generated when you access .output on an operator or use the TaskFlow API. The actual XCom value is resolved only when the downstream task executes, enabling dynamic DAG structures.
| Issue | Consequence | |-------|--------------| | DB becomes bottleneck | Many large XComs slow down scheduler | | Not designed for streaming | Only final values, not incremental | | No automatic cleanup (unless configured) | XCom rows accumulate | | Cross-DAG XCom is fragile | Requires manual conf passing | airflow xcom exclusive
Example:
The keyword "exclusive" in the context of XCom is not an official Airflow term, but it encapsulates several important characteristics that make XCom a unique and mechanism for data exchange. At its simplest, an XCom is a key-value
from airflow.decorators import dag, task from airflow.utils.dates import days_ago from airflow.models import XCom from airflow.utils.session import provide_session from datetime import datetime, timedelta @dag(start_date=datetime(2026, 1, 1), schedule="@weekly", catchup=False, tags=["maintenance"]) def xcom_cleanup_dag(): @task @provide_session def purge_old_xcoms(session=None): # Define retention boundary (e.g., delete xcoms older than 30 days) retention_date = datetime.now() - timedelta(days=30) deleted_rows = session.query(XCom).filter(XCom.timestamp < retention_date).delete(synchronize_session=False) print(f"Successfully purged deleted_rows older XCom rows from the database.") purge_old_xcoms() xcom_cleanup_dag() Use code with caution.
def pull_task(**context): value = context['ti'].xcom_pull(key='my_key', task_ids='push_task') # or pull all from previous task all_data = context['ti'].xcom_pull(task_ids='push_task') They are generated when you access
Most operators automatically push their execution result to this "reserved" key if do_xcom_push is enabled. Why "Exclusive" XComs Matter
The default XCom backend, BaseXCom , stores XComs directly in Airflow's metadata database. This design comes with a hard size limit: per XCom value. This is not an arbitrary number—it is enforced in the source code to prevent the metadata database from being overwhelmed by large data objects.