Building from source

Install vineyard

Vineyard is distributed as a python package and can be easily installed with pip:

pip3 install vineyard

Install etcd

Vineyard is based on etcd, please refer the doc to install it.

Install from source

Vineyard is open source on Github: https://github.com/v6d-io/v6d. You can obtain the source code using git:

git clone https://github.com/v6d-io/v6d
cd v6d
git submodule update --init

Prepare dependencies

Vineyard can be built and deployed on common Unix-like systems. Vineyard has been fully tests with C++ compilers that supports C++ 14.

Dependencies

Vineyard requires the following software as dependencies to build and run:

  • apache-arrow >= 3.0.0

  • gflags

  • glog

  • boost

  • mpi, for the graph data structure module

If you want to build the vineyard server, the following additional libraries are needed:

  • protobuf

  • grpc

And the following python packages are required:

  • libclang

    Can be installed using pip

    pip3 install libclang
    

Install on Ubuntu (or Debian)

Vineyard has been fully tested on Ubuntu 20.04. The dependencies can be installed by

apt-get install -y ca-certificates \
                   cmake \
                   doxygen \
                   libboost-all-dev \
                   libcurl4-openssl-dev \
                   libgflags-dev \
                   libgoogle-glog-dev \
                   libgrpc-dev \
                   libgrpc++-dev \
                   libmpich-dev \
                   libprotobuf-dev \
                   libssl-dev \
                   libunwind-dev \
                   libz-dev \
                   protobuf-compiler-grpc \
                   python3-pip \
                   wget

Then install the apache-arrow (see also https://arrow.apache.org/install):

wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb \
    -O /tmp/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
apt install -y -V /tmp/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
apt update -y
apt install -y libarrow-dev

Dependencies on MacOS

Vineyard has been tested on MacOS as well, the dependencies can be installed using brew:

brew install apache-arrow boost gflags glog grpc protobuf llvm mpich openssl zlib autoconf

Building vineyard

After the required dependencies are installed, you do an out-of-source build using CMake:

Tip

We recommend to use the brew installed LLVM as the compiler for building vineyard on MacOS, which can be accomplished by setting the environment variable CC and CXX:

export CC=$(brew --prefix llvm)/bin/clang
export CXX=$(brew --prefix llvm)/bin/clang++
mkdir build
cd build
cmake ..
make -j$(nproc)
sudo make install  # optionally

You will see vineyard server binary under the bin directory, and static or shared linked libraries will be placed under the lib-shared folder.

Building python wheels

After building the vineyard library successfully, you can package an install wheel distribution by

python3 setup.py bdist_wheel

Building the documentation

Vineyard documentation is organized and generated by sphinx. There are other packages that help us build the documentation, which can be easily installed using pip:

pip3 install -r requirements.txt -r requirements-dev.txt

Once installed, you could go to the docs/ directory and build the documentation by

cd docs/  # skip if you are already there
make html

Building on various platforms

Vineyard is continuously tested on various platforms and you may find building and installation steps from our CI: