Vineyard seamlessly integrates with the workflow orchestration engines, e.g., Apache Airflow and Kedro, enabling users to effortlessly incorporate Vineyard into their workflows for enhanced performance.
Moreover, the Airflow integration empowers users to work with large Python objects
featuring complex data types (e.g.,
pandas.DataFrame) at minimal cost, while
eliminating the need for cumbersome
The Kedro integration enables users to easily share large data objects across nodes in a pipeline and eliminates the high cost of (de)serialization and I/O compared with alternatives like AWS S3 or Minio, without the need to modify the pipeline code intrusively, and provides seamless user experience when scaling pipelines to Kubernetes.