We showcase step-by-step case studies of how to combine the functionalities of vineyard with existing data-intensive jobs. We show that vineyard can bring huge gains in both performance and conveniences when users have a complex workflow that involves multiple computing engines.

How vineyard can help in a distributed machine learning training workflow where various computing engine are involved.

Besides, vineyard has implemented a set of efficient data structures that needed in common data-intensive jobs, e.g., tensors, data frames, tables and graphs. The data types can be extended as well in a fairly straightforward way. Once registered the user defined custom types into the vineyard type registry, computing engines run on top of vineyard can immediately gain the benefits brought by vineyard.

Adding new data types and register to vineyard’s builder/resolver context.