Contributing to vineyard#

Vineyard has been developed by an active team of software engineers and researchers. Any contributions from the open-source community to improve this project are welcome!

Vineyard is licensed under Apache License 2.0.

Install development dependencies#

Vineyard requires the following C++ packages for development:

  • apache-arrow >= 0.17.1

  • gflags

  • glog

  • boost

  • gtest, for build test suites

  • protobuf

  • grpc

and the following python packages that can be easily installed using pip:

  • libclang

  • parsec

Developing vineyard using Docker#

To simplify the dependency installation process, we have provided a docker image with all requirements installed. The docker image can be found from vineyardcloudnative/vineyard-dev.

docker pull vineyardcloudnative/vineyard-dev:latest

Build the source#

You can do an out-of-source build using CMake:

mkdir build
cd build
cmake ..
make -j$(nproc)

The vineyardd target will be generated under the bin directory.

You may find building and installation instructions for other platforms from our CI:

Run unittests#

Vineyard has included a set of unittests in the continous integration process. Test cases can be built from the following command:

cd build
make vineyard_tests -j$(nproc)

Before running test cases, you need to ensure etcd is correctly installed, by brew install etcd on Mac or pip3 install etcd_distro on Linux distributions.

There’s a script to setup required environments and run test cases:

./test/runner.py --help
usage: runner.py [-h] [--with-cpp] [--with-python] [--with-io] [--with-deployment] [--with-migration] [--with-contrib] [--tests [TESTS [TESTS ...]]]

optional arguments:
  -h, --help            show this help message and exit
  --with-cpp            Whether to run C++ tests
  --with-python         Whether to run python tests
  --with-io             Whether to run IO adaptors tests
  --with-deployment     Whether to run deployment and scaling in/out tests
  --with-migration      Whether to run object migration tests
  --with-contrib        Whether to run python contrib tests
  --tests [TESTS [TESTS ...]]
                        Specify tests cases ro run

As shown above, you could run C++ unittests by

./test/runner --with-cpp

You could only run specified test case as well:

./test/runner --with-cpp --tests array_test dataframe_test

Documentation#

Documentation is generated using Doxygen and sphinx. Users can build vineyard’s documentation in the docs/ directory using:

cd docs/
make html

The HTML documentation will be available under docs/_build/html:

open _build/html/index.html

The latest version of online documentation can be found at https://v6d.io.

Vineyard provides comprehensive documents to explain the underlying design and implementation details. The documentation follows the syntax of Doxygen and sphinx markup. If you find anything you can help, submit pull request to us. Thanks for your enthusiasm!

Reporting bugs#

Vineyard is hosted on Github, and use Github issues as the bug tracker. You can file an issue when you find anything that is expected to work with vineyard but it doesn’t.

Before creating a new bug entry, we recommend you first search among existing vineyard bugs to see if it has already been resolved.

When creating a new bug entry, please provide necessary information of your problem in the description, such as operating system version, vineyard version, and other system configurations to help us diagnose the problem.

Submitting pull requests#

We also welcome any help on vineyard from the community, including but not limited to fixing bugs and adding new features. Contributors can follow this section for how to submit pull requests to vineyard:

Install pre-commit#

Vineyard use `pre-commit`_ to ensure no secrets are accidentally committed into the Git repository, you could first install `pre-commit`_ by

pip3 install pre-commit

The configure the necessary pre-commit hooks with

pre-commit install

Sign-off your commits#

Our project vineyard has enabled DCO thus you will be asked to sign-off your commits that are included in your pull requests. Git has a -s command line option that can sign-off your commit automatically:

git commit -s -m 'This is my commit message'

Code format#

Vineyard follows the Google C++ Style Guide. When submitting patches to vineyard, please format your code with clang-format by the Makefile command make vineyard_clformat, and make sure your code doesn’t break the cpplint convention using the CMakefile command make vineyard_cpplint.

Open a pull request#

When opening issues or submitting pull requests, we’ll ask you to prefix the pull request title with the issue number and the kind of patch (BUGFIX or FEATURE) in brackets, for example, [BUGFIX-1234] Fix crash in sealing vector to vineyard or [FEATURE-2345] Support seamless operability with PyTorch’s tensors.

Git workflow for newcomers#

You generally do NOT need to rebase your pull requests unless there are merge conflicts with the main. When Github complaining that “Can’t automatically merge” on your pull request, you’ll be asked to rebase your pull request on top of the latest main branch, using the following commands:

  • First rebasing to the most recent main:

    git remote add upstream https://github.com/v6d-io/v6d.git
    git fetch upstream
    git rebase upstream/main
    
  • Then git may show you some conflicts when it cannot merge, say conflict.cpp, you need - Manually modify the file to resolve the conflicts - After resolved, mark it as resolved by

    git add conflict.cpp
    
  • Then you can continue rebasing by

    git rebase --continue
    
  • Finally push to your fork, then the pull request will be got updated:

    git push --force
    

Cut a release#

The vineyard python package is built using the manylinux1 environments. The release version is built with Docker. The description of the base image can be found at docker/pypa/Dockerfile.manylinux1.