cassandra/.circleci

CircleCI config files

This directory contains the configuration for CircleCI continous integration platform. The file config.yml is the configuration file that is read by CircleCI. This file is automatically generated by the generate.sh script from the config-2_1.yml file.

The provided config.yml file uses low resources so users of the CircleCI free tier can use it. Additionally, there are three versions of this file using different resources so users who have access to premium CircleCI resources can use larger instances and more parallelism. These files are config.yml.LOWRES, config.yml.MIDRES and config.yml.HIGHRES. The default config.yml file is just a copy of config.yml.LOWRES.

Switching to higher resource settings

This directory contains generated files for low, middle and high resource settings. Switch between them by copying the correct file to config.yml and committing the result:

cp .circleci/config.yml.HIGHRES .circleci/config.yml

Alternatively, you can run the generate.sh script with the flags -l/-m/-h to regenerate the config.yml file from config-2_1.yml using LOWRES/MIDRES/HIGHRES. This script validates and applies any changes to the config-2_1.yml file, and it requires the CircleCI CLI to be installed.

Setting environment variables

Both config-2_1.yml and config.yml files contain a set of environment variables defining things like what dtest repo and branch to use, what tests could be repeatedly run, etc.

These environment variables can be directly edited in the config.yml file, although if you do this you should take into account that the entire set of env vars is repeated on every job.

A probably better approach is editing them in config-2_1.yml and then regenerate the config.yml file using the generate.sh script. You can also directly pass environment variable values to the generate.sh script with the -e flag. For example, to set the dtest repo and branch with MIDRES config you can run:

generate.sh -m \
  -e DTEST_REPO=https://github.com/adelapena/cassandra-dtest.git \
  -e DTEST_BRANCH=CASSANDRA-8272

Running tests in a loop

Running the generate.sh script with use git diff to find the new or modified tests. The script will then create jobs to run each of these new or modified tests for a certain number of times, to verify that they are stable. You can use environment variables to specify the number of iterations of each type of test:

generate.sh -m \
  -e REPEATED_UTESTS_COUNT=500 \
  -e REPEATED_UTESTS_FQLTOOL_COUNT=500 \
  -e REPEATED_UTESTS_LONG_COUNT=100 \
  -e REPEATED_UTESTS_STRESS_COUNT=500 \
  -e REPEATED_SIMULATOR_DTESTS_COUNT=500 \
  -e REPEATED_JVM_DTESTS_COUNT=500 \
  -e REPEATED_JVM_UPGRADE_DTESTS_COUNT=500 \
  -e REPEATED_DTESTS_COUNT=500 \
  -e REPEATED_LARGE_DTESTS_COUNT=100 \
  -e REPEATED_UPGRADE_DTESTS_COUNT=25 \
  -e REPEATED_ANT_TEST_COUNT=500

You can also specify whether the iteration should fail on the first test failure:

generate.sh -m -e REPEATED_TESTS_STOP_ON_FAILURE=false

In addition to the automatically detected tests, it's also possible to provide lists of specific tests to be repeated:

generate.sh -m \
  -e REPEATED_UTESTS=org.apache.cassandra.cql3.ViewTest,org.apache.cassandra.db.CellTest \
  -e REPEATED_UTESTS_FQLTOOL=org.apache.cassandra.fqltool.FQLCompareTest \
  -e REPEATED_UTESTS_LONG=org.apache.cassandra.io.sstable.CQLSSTableWriterLongTest#testWideRow \
  -e REPEATED_UTESTS_STRESS=org.apache.cassandra.stress.generate.DistributionGaussianTest \
  -e REPEATED_SIMULATOR_DTESTS=org.apache.cassandra.simulator.test.TrivialSimulationTest \
  -e REPEATED_DTESTS=cql_test.py,consistency_test.py::TestAvailability::test_simple_strategy \
  -e REPEATED_LARGE_DTESTS=consistency_test.py::TestAvailability::test_network_topology_strategy \
  -e REPEATED_JVM_DTESTS=org.apache.cassandra.distributed.test.PagingTest#testPaging \
  -e REPEATED_UPGRADE_DTESTS=upgrade_tests/cql_tests.py \
  -e REPEATED_JVM_UPGRADE_DTESTS=org.apache.cassandra.distributed.upgrade.GroupByTest

For particular Ant test targets that are not included in the regular test suites, you can use the run_repeated_utest job:

generate.sh -m \
  -e REPEATED_ANT_TEST_TARGET=test-cdc \
  -e REPEATED_ANT_TEST_CLASS=org.apache.cassandra.cql3.ViewTest \
  -e REPEATED_ANT_TEST_METHODS=testCompoundPartitionKey,testStaticTable \
  -e REPEATED_ANT_TEST_VNODES=false \
  -e REPEATED_ANT_TEST_COUNT=500

Putting all together, you can have runs as complex as:

generate.sh -m \
  -e REPEATED_TESTS_STOP_ON_FAILURE=true \
  -e REPEATED_UTESTS=org.apache.cassandra.cql3.ViewTest,org.apache.cassandra.db.CellTest \
  -e REPEATED_UTESTS_COUNT=500 \
  -e REPEATED_UTESTS_FQLTOOL=org.apache.cassandra.fqltool.FQLCompareTest \
  -e REPEATED_UTESTS_FQLTOOL_COUNT=500 \
  -e REPEATED_UTESTS_LONG=org.apache.cassandra.io.sstable.CQLSSTableWriterLongTest#testWideRow \
  -e REPEATED_UTESTS_LONG_COUNT=100 \
  -e REPEATED_UTESTS_STRESS=org.apache.cassandra.stress.generate.DistributionGaussianTest \
  -e REPEATED_UTESTS_STRESS_COUNT=500 \
  -e REPEATED_SIMULATOR_DTESTS=org.apache.cassandra.simulator.test.TrivialSimulationTest \
  -e REPEATED_SIMULATOR_DTESTS_COUNT=500 \
  -e REPEATED_DTESTS=cql_test.py,consistency_test.py::TestAvailability::test_simple_strategy \
  -e REPEATED_DTESTS_COUNT=500 \
  -e REPEATED_LARGE_DTESTS=consistency_test.py,materialized_views_test.py \
  -e REPEATED_LARGE_DTESTS=100 \
  -e REPEATED_JVM_DTESTS=org.apache.cassandra.distributed.test.PagingTest#testPaging \
  -e REPEATED_JVM_DTESTS_COUNT=500 \
  -e REPEATED_UPGRADE_DTESTS=upgrade_tests/cql_tests.py \
  -e REPEATED_UPGRADE_DTESTS_COUNT=25 \
  -e REPEATED_JVM_UPGRADE_DTESTS=org.apache.cassandra.distributed.upgrade.GroupByTest \
  -e REPEATED_JVM_UPGRADE_DTESTS_COUNT=500 \
  -e REPEATED_ANT_TEST_TARGET=test-cdc \
  -e REPEATED_ANT_TEST_CLASS=org.apache.cassandra.cql3.ViewTest \
  -e REPEATED_ANT_TEST_METHODS=testCompoundPartitionKey,testStaticTable \
  -e REPEATED_ANT_TEST_VNODES=false \
  -e REPEATED_ANT_TEST_COUNT=500

Updating the config

For configuration changes meant to be permanent in the Apache repo you should never edit the config.yml file manually. Instead, you should edit the config-2_1.yml file and then regenerate the config.yml, config.yml.LOWRES, config.yml.MIDRES and config.yml.HIGHRES files by runnining the generate.sh script with -a flag. For using this script you need to install the CircleCI CLI.

As for temporal changes done while working in a patch, such as pointing to you dtest repo or running a test repeatedly, you can either directly edit config.yml or edit config-2_1.yml and then regenerate config.yml with the generate.sh script using a -l/-m/-h flag. When this flag is used only the config.yml will be generated.

Please note that any previous swapping or edition of the generated files will be overriden by running generate.sh with -a argument, returning config.yml to the default LOWRES. So if you previously swapped your config.yml to MIDRES or HIGHRES you would need to either swap it again or use the -l/-m/-h script flags.

Read below for details how to generate the files manually without the generate.sh script:

1. make your edits to config-2_1.yml - let it stay at lowres settings
2. generate a valid LOWRES file: circleci config process config-2_1.yml > config.yml.LOWRES
3. add the Apache license header to the newly created LOWRES file: cat license.yml config.yml.LOWRES > config.yml.LOWRES.new && mv config.yml.LOWRES.new config.yml.LOWRES
4. then apply the highres patch to config-2_1.yml; patch -o config-2_1.yml.HIGHRES config-2_1.yml config-2_1.yml.high_res.patch (this creates a new file config-2_1.yml.HIGHRES instead of in-place patching config-2_1.yml) Note that if the patch no longer applies to config-2_1.yml a new patch file is needed, do this by manually making config-2_1.yml high resource and create the patch file based on the diff (don't commit it though).
5. generate the HIGHRES file: circleci config process config-2_1.yml.HIGHRES > config.yml.HIGHRES
6. remove the temporary patched HIGHRES file: rm config-2_1.yml.HIGHRES
7. add the Apache license header to the newly created HIGHRES file: cat license.yml config.yml.HIGHRES > config.yml.HIGHRES.new && mv config.yml.HIGHRES.new config.yml.HIGHRES
8. repeat the last steps to generate the MIDRES file:

   patch -o config-2_1.yml.MIDRES config-2_1.yml config-2_1.yml.mid_res.patch
   circleci config process config-2_1.yml.MIDRES > config.yml.MIDRES
   rm config-2_1.yml.MIDRES
   cat license.yml config.yml.MIDRES > config.yml.MIDRES.new && mv config.yml.MIDRES.new config.yml.MIDRES
   

9. finally, remember to update the config.yml