Workflow orchestration#
Vineyard seamlessly integrates with the workflow orchestration engine, Apache Airflow,
enabling users to effortlessly incorporate Vineyard into their workflows for enhanced performance.
Moreover, the Airflow integration empowers users to work with large Python objects
featuring complex data types (e.g., pandas.DataFrame
) at minimal cost, while
eliminating the need for cumbersome pickle.dump/loads
operations.
Airflow uses vineyard as the XCom backend to efficiently handle complex data in Python.