Move build and test scripts in-tree, under .build/

See .build/README.md  .Build and test scripts cover all CI activities in an CI agnostic manner. Scripts are found in non-docker and dockerised variants.

The non-docker scripts are found first under .build/ .These interact directly with ant, providing additional checks and environment setup. It is intended that what can be pushed down into the base ant build layer should be. The
debian and redhat build scripts are hidden under .build/docker because they are generally not usuable outside of their corresponding linux dists, and we don't want to encourage anything to the contrary.  Bring the artifact/deb/rpm
build scripts (and associated docker images) from cassandra-builds repo to the .build directory.  These packaging script for debian and redhat are now separated from the artifacts scripts. An additional build script check-code.sh
has been added that is solely responsible for all linter and code checks (includes owasp dependency checker).  Only JDK11 and JDK17 are supported (as JDK8 is soon to be dropped).  Scripts do not clean (or realclean) so that
scripts can be better pipelined, but `ant jar` was required to be added to circleci and ci-cassandra.a.o cqlsh-tests.  Scripts can define a custom BUILD_DIR (build.dir) which can permit running parrallel builds off the same source
(though not all tests support this, or their support for it has atrophied).  All build and test artifacts, results, and logs are found under build/ (or the BUILD_DIR).  The build local venv and .ccm directories are now also
created under build (or BUILD_DIR).  The test venv, logs and results are wiped each run.  Use a separate mktemp directory each run, under /tmp (respect /tmp responsibility, and python fails when tmpdir is on a docker volume).
Don't set CASSANDRA_HOME when running python dtests (and fail-fast if `ant artifacts` was used to build instead of `ant jar`).

The docker scripts are found lower down under .build/docker/ .These scripts re-used the non-docker scripts inside containers. The docker images are versioned controlled in the same directory as dockerfiles, and tagged by their
md5sums.  When running the docker scripts these docker images will, if not in the local registry, will first be attempted to be pulled from dockerhub and only then fallback to be built locally.  This allows the dockerfiles to be
modified in existing patches and be run in existing CI as-is.  Docker scripts timeout after one hour.  virtualenv-clone is used, re-using the python versioned venvs in the image saves time over creating new ones each run.  The inner-spliting of docker containers has been removed.  It added a lot of complexity, for the most part duplicating the splitting concept, for little performance gain.

Ant target 'generate-unified-test-report' added, used for aggregating test reports and printing a summary.

 patch by Mick Semb Wever; reviewed by Brandon Williams, Josh McKenzie, Maxim Muzafarov, Stefan Miklosovic for CASSANDRA-18133

.build/README.md

@@ -0,0 +1,130 @@
+Building and Testing with the helper sripts
+-------------------------------------------
+
+Information on building and testing beyond the use of ant.
+All scripts also print help if the first argument is `-h`.
+
+Code Checks and Lints
+---------------------
+
+Run in docker:
+
+    .build/docker/check-code.sh
+
+
+Run without docker:
+
+    .build/check-code.sh
+
+
+Run in docker with a specific jdk.
+The following applies to all build scripts.
+
+    .build/docker/check-code.sh 11
+
+
+Run in docker with a specific build path.
+This permits parallel builds off the same source path.
+The following applies to all build scripts.
+
+    build_dir=/tmp/cass_Mtu462n .build/docker/check-code.sh
+
+
+Building Artifacts (tarball and maven)
+-------------------------------------
+
+Build with docker:
+
+    .build/docker/build-artifacts.sh
+
+
+Build without docker:
+
+    .build/build-artifacts.sh
+
+
+Build in docker with a specific jdk:
+
+    .build/docker/build-artifacts.sh 11
+
+
+Building Debian and RedHat packages
+-----------------------------------
+
+The packaging scripts are only intended to be used with docker.
+
+Build:
+
+    .build/docker/build-debian.sh
+    .build/docker/build-redhat.sh
+
+
+Build with a specific jdk:
+
+    .build/docker/build-debian.sh 11
+    .build/docker/build-redhat.sh rpm 11
+
+
+Build with centos7 and a specific jdk:
+
+    .build/docker/build-redhat.sh noboolean 11
+
+
+Running Tests
+-------------
+
+Running unit tests with docker:
+
+    .build/docker/run-tests.sh test
+
+
+Running unittests without docker:
+
+    .build/run-tests.sh test
+
+
+Running only a split of unittests, with docker:
+
+    .build/docker/run-tests.sh test 1/64
+
+
+Running unittests with a specific jdk with docker:
+
+    .build/docker/run-tests.sh test 1/64 11
+
+
+Running only unit tests matching a regexp, with docker:
+
+    .build/docker/run-tests.sh test VerifyTest 11
+
+
+Running other types of tests with docker:
+
+    .build/docker/run-tests.sh test
+    .build/docker/run-tests.sh stress-test
+    .build/docker/run-tests.sh fqltool-test
+    .build/docker/run-tests.sh microbench
+    .build/docker/run-tests.sh test-cdc
+    .build/docker/run-tests.sh test-compression
+    .build/docker/run-tests.sh test-burn
+    .build/docker/run-tests.sh long-test
+    .build/docker/run-tests.sh cqlsh-test
+    .build/docker/run-tests.sh jvm-dtest
+    .build/docker/run-tests.sh jvm-dtest-upgrade
+    .build/docker/run-tests.sh dtest
+    .build/docker/run-tests.sh dtest-novnode
+    .build/docker/run-tests.sh dtest-offheap
+    .build/docker/run-tests.sh dtest-large
+    .build/docker/run-tests.sh dtest-large-novnode
+    .build/docker/run-tests.sh dtest-upgrade
+
+
+Running python dtests without docker:
+
+    .build/run-python-dtests.sh dtest
+
+
+Other test types without docker:
+
+    .build/run-tests.sh test
+

.build/build-artifacts.sh

@@ -0,0 +1,28 @@
+#!/bin/sh -e
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# variables, with defaults
+[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)"
+
+# pre-conditions
+command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; }
+[ -d "${CASSANDRA_DIR}" ] || { echo >&2 "Directory ${CASSANDRA_DIR} must exist"; exit 1; }
+[ -f "${CASSANDRA_DIR}/build.xml" ] || { echo >&2 "${CASSANDRA_DIR}/build.xml must exist"; exit 1; }
+
+# execute
+ant -f "${CASSANDRA_DIR}/build.xml" artifacts -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true
+exit $?

.build/build-git.xml

@@ -18,7 +18,8 @@
 -->
 <project basedir="." name="apache-cassandra-git-tasks"
          xmlns:if="ant:if">
-    <target name="get-git-sha">
+
+    <target name="_get-git-sha">
         <exec executable="git" osfamily="unix" dir="${basedir}" logError="false" failonerror="false" failifexecutionfails="false" resultproperty="git.is-available.exit-code">
             <arg value="rev-parse"/>
             <arg value="--is-inside-work-tree"/>

.build/build-jars.sh

@@ -0,0 +1,30 @@
+#!/bin/sh -e
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# temporary between CASSANDRA-18133 and CASSANDRA-18594
+
+# variables, with defaults
+[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)"
+
+# pre-conditions
+command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; }
+[ -d "${CASSANDRA_DIR}" ] || { echo >&2 "Directory ${CASSANDRA_DIR} must exist"; exit 1; }
+[ -f "${CASSANDRA_DIR}/build.xml" ] || { echo >&2 "${CASSANDRA_DIR}/build.xml must exist"; exit 1; }
+
+# execute
+ant -f "${CASSANDRA_DIR}/build.xml" jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Dno-javadoc=true
+exit $?

.build/build-owasp.xml

@@ -18,14 +18,14 @@
   -->
 <project basedir="." name="apache-cassandra-owasp-tasks">
     <property name="dependency-check.version" value="8.3.1"/>
-    <property name="dependency-check.home" value="${build.dir}/dependency-check-ant-${dependency-check.version}"/>
+    <property name="dependency-check.home.base" value="${build.dir}"/>
+    <property name="dependency-check.home" value="${dependency-check.home.base}/dependency-check-ant-${dependency-check.version}"/>
 
     <condition property="is.dependency.check.jar">
         <available file="${dependency-check.home}/dependency-check-ant/dependency-check-ant.jar" type="file" />
     </condition>
 
     <target name="dependency-check-download"
-            depends="build"
             description="Fetch OWASP Dependency checker"
             unless="is.dependency.check.jar">
 
@@ -33,8 +33,10 @@
 
         <mkdir dir="${dependency-check.home}"/>
 
-        <get src="https://github.com/jeremylong/DependencyCheck/releases/download/v${dependency-check.version}/dependency-check-ant-${dependency-check.version}-release.zip"
+        <retry retrycount="3" retrydelay="10" >
-             dest="${dependency-check.home}/dependency-check-ant-${dependency-check.version}-release.zip"/>
+            <get src="https://github.com/jeremylong/DependencyCheck/releases/download/v${dependency-check.version}/dependency-check-ant-${dependency-check.version}-release.zip"
+                 dest="${dependency-check.home}/dependency-check-ant-${dependency-check.version}-release.zip"/>
+        </retry>
 
         <unzip src="${dependency-check.home}/dependency-check-ant-${dependency-check.version}-release.zip" dest="${dependency-check.home}"/>
     </target>

.build/build-rat.xml

@@ -86,10 +86,10 @@
                  <!-- Documentation files -->
                  <exclude name="**/doc/modules/**/*"/>
                  <exclude name="**/src/java/**/*.md"/>
-                 <!-- NOTICE files -->
+                 <!-- NOTICE, LICENSE, README files -->
                  <exclude name="**/NOTICE.md"/>
-                 <!-- LICENSE files -->
                  <exclude name="**/LICENSE.md"/>
+                 <exclude name="**/README.md"/>
             </fileset>
         </rat:report>
         <exec executable="grep" outputproperty="rat.failed.files" failifexecutionfails="false">
@@ -105,15 +105,4 @@
             </condition>
         </fail>
     </target>
-
-    <target name="_assert_rat_output">
-        <fail message="The rat report at build/rat.txt was not generated. Please ensure that the rat-check task is able to run successfully. For dev builds only, touch build/rat.txt to skip this check">
-            <condition>
-                <not>
-                    <available file="${build.dir}/rat.txt" />
-                </not>
-            </condition>
-        </fail>
-    </target>
-
 </project>

.build/build-resolver.xml

@@ -46,8 +46,10 @@
         <echo>Downloading Resolver ANT Tasks...</echo>
         <mkdir dir="${local.repository}/org/apache/maven/resolver/maven-resolver-ant-tasks/${resolver-ant-tasks.version}" />
 
-        <get src="${resolver-ant-tasks.url}/${resolver-ant-tasks.version}/maven-resolver-ant-tasks-${resolver-ant-tasks.version}-uber.jar"
+        <retry retrycount="3" retrydelay="10" >
-             dest="${resolver-ant-tasks.local}" usetimestamp="true" quiet="true"/>
+            <get src="${resolver-ant-tasks.url}/${resolver-ant-tasks.version}/maven-resolver-ant-tasks-${resolver-ant-tasks.version}-uber.jar"
+                 dest="${resolver-ant-tasks.local}" usetimestamp="true" quiet="true"/>
+        </retry>
     </target>
 
     <target name="resolver-init" depends="init,_resolver_download" unless="resolver-ant-tasks.initialized" description="Initialize Resolver ANT Tasks">
@@ -175,15 +177,19 @@
         <resolvepom file="${build.dir}/${final.name}.pom" id="all-pom" />
         <resolvepom file="${build.dir}/tmp-${final.name}-deps.pom" id="pom-deps" />
 
-        <resolve>
+        <retry retrycount="3" retrydelay="10" >
-            <dependencies pomRef="all-pom"/>
+            <resolve>
-            <files dir="${build.dir.lib}/jars" layout="{artifactId}-{version}-{classifier}.{extension}" scopes="compile,provided,!system"/>
+                <dependencies pomRef="all-pom"/>
-        </resolve>
+                <files dir="${build.dir.lib}/jars" layout="{artifactId}-{version}-{classifier}.{extension}" scopes="compile,provided,!system"/>
-        <resolve>
+            </resolve>
-            <dependencies pomRef="pom-deps"/>
+        </retry>
-            <!-- Needed to include compile here, so ant _build-test would not fail on missing jimfs dependency -->
+        <retry retrycount="3" retrydelay="10" >
-            <files dir="${test.lib}/jars" layout="{artifactId}-{version}-{classifier}.{extension}" scopes="compile,test,!provide,!system"/>
+            <resolve>
-        </resolve>
+                <dependencies pomRef="pom-deps"/>
+                <!-- Needed to include compile here, so ant _build-test would not fail on missing jimfs dependency -->
+                <files dir="${test.lib}/jars" layout="{artifactId}-{version}-{classifier}.{extension}" scopes="compile,test,!provide,!system"/>
+            </resolve>
+        </retry>
 
 
         <!-- jacoco agent jar comes wrapped in a jar -->
@@ -198,44 +204,19 @@
     <target name="resolver-dist-lib" depends="resolver-retrieve-build">
         <resolvepom file="${build.dir}/${final.name}.pom" id="all-pom" />
 
-        <resolve failOnMissingAttachments="true">
+        <retry retrycount="3" retrydelay="10" >
-            <dependencies pomRef="all-pom"/>
+            <resolve failOnMissingAttachments="true">
-            <files dir="${build.lib}" layout="{artifactId}-{version}-{classifier}.{extension}" scopes="compile,!provide,!system"/>
+                <dependencies pomRef="all-pom"/>
-        </resolve>
+                <files dir="${build.lib}" layout="{artifactId}-{version}-{classifier}.{extension}" scopes="compile,!provide,!system"/>
+            </resolve>
+        </retry>
         <mkdir dir="${local.repository}/org/apache/cassandra/deps/sigar-bin"/>
         <mkdir dir="${build.lib}/sigar-bin"/>
 
-        <!-- files.pythonhosted.org/packages -->
+        <retry retrycount="3" retrydelay="10" >
-        <get src="${artifact.python.pypi}/59/a0/cf4cd997e1750f0c2d91c6ea5abea218251c43c3581bcc2f118b00baf5cf/futures-2.1.6-py2.py3-none-any.whl" dest="${local.repository}/org/apache/cassandra/deps/futures-2.1.6-py2.py3-none-any.zip" usetimestamp="true" quiet="true" skipexisting="true"/>
+            <antcall target="_resolver-dist-lib_get_files"/>
-        <get src="${artifact.python.pypi}/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl" dest="${local.repository}/org/apache/cassandra/deps/six-1.12.0-py2.py3-none-any.zip" usetimestamp="true" quiet="true" skipexisting="true"/>
+        </retry>
-        <get src="${artifact.python.pypi}/37/b2/ef1124540ee2c0b417be8d0f74667957e6aa084a3f26621aa67e2e77f3fb/pure_sasl-0.6.2-py2-none-any.whl" dest="${local.repository}/org/apache/cassandra/deps/pure_sasl-0.6.2-py2-none-any.zip" usetimestamp="true" quiet="true" skipexisting="true"/>
-        <get src="${artifact.python.pypi}/59/7c/e39aca596badaf1b78e8f547c807b04dae603a433d3e7a7e04d67f2ef3e5/wcwidth-0.2.5-py2.py3-none-any.whl" dest="${local.repository}/org/apache/cassandra/deps/wcwidth-0.2.5-py2.py3-none-any.zip" usetimestamp="true" quiet="true" skipexisting="true"/>
 
-        <!-- apache/cassandra/lib -->
-        <get src="${lib.download.base.url}/lib/geomet-0.1.0.zip" dest="${local.repository}/org/apache/cassandra/deps/geomet-0.1.0.zip" usetimestamp="true" quiet="true" skipexisting="true"/>
-        <get dest="${local.repository}/org/apache/cassandra/deps/sigar-bin/" quiet="true" usetimestamp="true" skipexisting="true">
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-amd64-freebsd-6.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-amd64-linux.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-amd64-solaris.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ia64-hpux-11.sl"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ia64-linux.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-pa-hpux-11.sl"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ppc-aix-5.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ppc-linux.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ppc64-aix-5.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ppc64-linux.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ppc64le-linux.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-s390x-linux.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-sparc-solaris.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-sparc64-solaris.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-universal-macosx.dylib"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-universal64-macosx.dylib"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-x86-freebsd-5.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-x86-freebsd-6.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-x86-linux.so"/>
-            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-x86-solaris.so"/>
-        </get>
-        
         <copy todir="${build.lib}" quiet="true">
             <file file="${local.repository}/org/apache/cassandra/deps/futures-2.1.6-py2.py3-none-any.zip"/>
             <file file="${local.repository}/org/apache/cassandra/deps/six-1.12.0-py2.py3-none-any.zip"/>
@@ -266,4 +247,38 @@
             <file file="${local.repository}/org/apache/cassandra/deps/sigar-bin/libsigar-x86-solaris.so"/>
         </copy>
     </target>
+
+    <target name="_resolver-dist-lib_get_files">
+        <!-- files.pythonhosted.org/packages -->
+        <get src="${artifact.python.pypi}/59/a0/cf4cd997e1750f0c2d91c6ea5abea218251c43c3581bcc2f118b00baf5cf/futures-2.1.6-py2.py3-none-any.whl" dest="${local.repository}/org/apache/cassandra/deps/futures-2.1.6-py2.py3-none-any.zip" usetimestamp="true" quiet="true" skipexisting="true"/>
+        <get src="${artifact.python.pypi}/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl" dest="${local.repository}/org/apache/cassandra/deps/six-1.12.0-py2.py3-none-any.zip" usetimestamp="true" quiet="true" skipexisting="true"/>
+        <get src="${artifact.python.pypi}/37/b2/ef1124540ee2c0b417be8d0f74667957e6aa084a3f26621aa67e2e77f3fb/pure_sasl-0.6.2-py2-none-any.whl" dest="${local.repository}/org/apache/cassandra/deps/pure_sasl-0.6.2-py2-none-any.zip" usetimestamp="true" quiet="true" skipexisting="true"/>
+        <get src="${artifact.python.pypi}/59/7c/e39aca596badaf1b78e8f547c807b04dae603a433d3e7a7e04d67f2ef3e5/wcwidth-0.2.5-py2.py3-none-any.whl" dest="${local.repository}/org/apache/cassandra/deps/wcwidth-0.2.5-py2.py3-none-any.zip" usetimestamp="true" quiet="true" skipexisting="true"/>
+
+        <!-- apache/cassandra/lib -->
+        <get src="${lib.download.base.url}/lib/geomet-0.1.0.zip" dest="${local.repository}/org/apache/cassandra/deps/geomet-0.1.0.zip" usetimestamp="true" quiet="true" skipexisting="true"/>
+        <get dest="${local.repository}/org/apache/cassandra/deps/sigar-bin/" quiet="true" usetimestamp="true" skipexisting="true">
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-amd64-freebsd-6.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-amd64-linux.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-amd64-solaris.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ia64-hpux-11.sl"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ia64-linux.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-pa-hpux-11.sl"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ppc-aix-5.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ppc-linux.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ppc64-aix-5.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ppc64-linux.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-ppc64le-linux.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-s390x-linux.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-sparc-solaris.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-sparc64-solaris.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-universal-macosx.dylib"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-universal64-macosx.dylib"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-x86-freebsd-5.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-x86-freebsd-6.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-x86-linux.so"/>
+            <url url="${lib.download.base.url}/lib/sigar-bin/libsigar-x86-solaris.so"/>
+        </get>
+    </target>
+
 </project>

.build/check-code.sh

@@ -0,0 +1,28 @@
+#!/bin/sh -e
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# variables, with defaults
+[ "x${CASSANDRA_DIR}" != "x" ] || { CASSANDRA_DIR="$(dirname "$0")/.."; }
+
+# pre-conditions
+command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; }
+[ -d "${CASSANDRA_DIR}" ] || { echo >&2 "Directory ${CASSANDRA_DIR} must exist"; exit 1; }
+[ -f "${CASSANDRA_DIR}/build.xml" ] || { echo >&2 "${CASSANDRA_DIR}/build.xml must exist"; exit 1; }
+
+# execute
+ant -f "${CASSANDRA_DIR}/build.xml" rat-check eclipse-warnings checkstyle checkstyle-test dependency-check
+exit $?

.build/docker/_build-debian.sh

@@ -0,0 +1,129 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+################################
+#
+# Prep
+#
+################################
+
+# variables, with defaults
+[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)"
+[ "x${DIST_DIR}" != "x" ] || DIST_DIR="${CASSANDRA_DIR}/build"
+
+# pre-conditions
+command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; }
+command -v git >/dev/null 2>&1 || { echo >&2 "git needs to be installed"; exit 1; }
+command -v dch >/dev/null 2>&1 || { echo >&2 "dch needs to be installed"; exit 1; }
+command -v dpkg-parsechangelog >/dev/null 2>&1 || { echo >&2 "dpkg-parsechangelog needs to be installed"; exit 1; }
+command -v dpkg-buildpackage >/dev/null 2>&1 || { echo >&2 "dpkg-buildpackage needs to be installed"; exit 1; }
+[ -d "${CASSANDRA_DIR}" ] || { echo >&2 "Directory ${CASSANDRA_DIR} must exist"; exit 1; }
+[ -f "${CASSANDRA_DIR}/build.xml" ] || { echo >&2 "${CASSANDRA_DIR}/build.xml must exist"; exit 1; }
+[ -d "${DIST_DIR}" ] || mkdir -p "${DIST_DIR}"
+
+################################
+#
+# Main
+#
+################################
+
+set -e
+
+# note, this edits files in your working cassandra directory
+pushd $CASSANDRA_DIR >/dev/null
+export BUILD_DIR="$(realpath --relative-to=$CASSANDRA_DIR ${DIST_DIR})"
+
+# Used version for build will always depend on the git referenced used for checkout above
+# Branches will always be created as snapshots, while tags are releases
+tag=`git describe --tags --exact-match 2>/dev/null || true`
+branch=`git symbolic-ref -q --short HEAD 2>/dev/null || true`
+
+is_tag=false
+git_version=''
+
+# Parse version from build.xml so we can verify version against release tags and use the build.xml version
+# for any branches. Truncate from snapshot suffix if needed.
+buildxml_version=`grep 'property\s*name="base.version"' build.xml |sed -ne 's/.*value="\([^"]*\)".*/\1/p'`
+regx_snapshot="([0-9.]+)-SNAPSHOT$"
+if [[ $buildxml_version =~ $regx_snapshot ]]; then
+   buildxml_version=${BASH_REMATCH[1]}
+fi
+
+if [ "$tag" ]; then
+   is_tag=true
+   # Official release
+   regx_tag="cassandra-(([0-9.]+)(-(alpha|beta|rc)[0-9]+)?)$"
+   # Tentative release
+   regx_tag_tentative="(([0-9.]+)(-(alpha|beta|rc)[0-9]+)?)-tentative$"
+   if [[ $tag =~ $regx_tag ]] || [[ $tag =~ $regx_tag_tentative ]]; then
+      git_version=${BASH_REMATCH[1]}
+   else
+      echo "Error: could not recognize version from tag $tag">&2
+      exit 2
+   fi
+   CASSANDRA_VERSION=$git_version
+   CASSANDRA_REVISION='1'
+else
+   regx_branch="cassandra-([0-9.]+)$"
+   if [[ $branch =~ $regx_branch ]]; then
+      git_version=${BASH_REMATCH[1]}
+   else
+      # This could be either trunk or any dev branch or SHA, so we won't be able to get the version
+      # from the branch name. In this case, fall back to debian change log version.
+      git_version=$(dpkg-parsechangelog | sed -ne 's/^Version: \(.*\).*/\1/p' | sed 's/~/-/')
+      if [ -z $git_version ]; then
+         echo "Error: could not recognize version from branch $branch">&2
+         exit 2
+      else
+         echo "Warning: could not recognize version from branch. dpkg version is $git_version"
+      fi
+   fi
+    # if CASSANDRA_VERSION is -alphaN, -betaN, -rcN, then rpmbuild fails on the '-' char; replace with '~'
+    CASSANDRA_VERSION=${buildxml_version/-/\~}
+    dt=`date +"%Y%m%d"`
+    ref=`git rev-parse --short HEAD`
+    CASSANDRA_REVISION="${dt}git${ref}"
+    dch -D unstable -v "${CASSANDRA_VERSION}-${CASSANDRA_REVISION}" --package "cassandra" "building ${CASSANDRA_VERSION}-${CASSANDRA_REVISION}"
+fi
+
+# The version used for the deb build process will the current version in the debian/changelog file.
+# See debian/rules for how the value is read. The only thing left for us to do here is to check if
+# the changes file contains the correct version for the checked out git revision. The version value
+# has to be updated manually by a committer and we only warn and abort on mismatches here.
+changelog_version=$(dpkg-parsechangelog | sed -ne 's/^Version: \(.*\).*/\1/p' | sed 's/~/-/')
+chl_expected="${buildxml_version}"
+if [[ ! $changelog_version =~ $chl_expected ]]; then
+   echo "Error: changelog version (${changelog_version}) doesn't match expected (${chl_expected})">&2
+   exit 3
+fi
+
+# Version (base.version) in build.xml must be set manually as well. Let's validate the set value.
+if [ $buildxml_version != $git_version ]; then
+   echo "Warning: build.xml version ($buildxml_version) not matching git/dpkg derived version ($git_version)">&2
+fi
+
+# build package
+dpkg-buildpackage -rfakeroot -uc -us -tc --source-option=--tar-ignore=.git
+
+# Move created artifacts to dist dir mapped to docker host directory (must have proper permissions)
+mv ../cassandra[-_]*${CASSANDRA_VERSION}-* "${DIST_DIR}"
+# clean build deps
+rm -f cassandra-build-deps_*
+# restore debian/changelog
+git restore debian/changelog
+
+popd >/dev/null

.build/docker/_build-redhat.sh

@@ -0,0 +1,122 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+################################
+#
+# Prep
+#
+################################
+
+# variables, w/ defaults, w/ checks
+[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)"
+[ "x${DIST_DIR}" != "x" ] || DIST_DIR="${CASSANDRA_DIR}/build"
+[ "x${RPM_BUILD_DIR}" != "x" ] || RPM_BUILD_DIR="$(mktemp -d /tmp/rpmbuild.XXXXXX)"
+
+# pre-conditions
+command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; }
+command -v git >/dev/null 2>&1 || { echo >&2 "git needs to be installed"; exit 1; }
+command -v rpmbuild >/dev/null 2>&1 || { echo >&2 "rpm-build needs to be installed"; exit 1; }
+[ -d "${CASSANDRA_DIR}" ] || { echo >&2 "Directory ${CASSANDRA_DIR} must exist"; exit 1; }
+[ -f "${CASSANDRA_DIR}/build.xml" ] || { echo >&2 "${CASSANDRA_DIR}/build.xml must exist"; exit 1; }
+[ -d "${DIST_DIR}" ] || mkdir -p "${DIST_DIR}"
+[ -d "${RPM_BUILD_DIR}/SOURCES" ] || mkdir -p ${RPM_BUILD_DIR}/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
+
+
+if [ "$1" == "-h" ]; then
+   echo "$0 [-h] [dist_type]"
+   echo "dist types are [rpm, noboolean] and rpm is default"
+   exit 1
+fi
+
+RPM_DIST=$1
+[ "x${RPM_DIST}" != "x" ] || RPM_DIST="rpm"
+
+if [ "${RPM_DIST}" == "rpm" ]; then
+   RPM_SPEC="redhat/cassandra.spec"
+elif [ "${RPM_DIST}" == "noboolean" ]; then # noboolean
+   RPM_SPEC="redhat/noboolean/cassandra.spec"
+else
+   echo >&2 "Only rpm and noboolean are valid dist_type arguments. Got ${RPM_DIST}"
+   exit 1
+fi
+
+################################
+#
+# Main
+#
+################################
+
+set -e
+
+# note, this edits files in your working cassandra directory
+pushd $CASSANDRA_DIR >/dev/null
+
+# Used version for build will always depend on the git referenced used for checkout above
+# Branches will always be created as snapshots, while tags are releases
+tag=`git describe --tags --exact-match 2> /dev/null || true`
+branch=`git symbolic-ref -q --short HEAD 2> /dev/null || true`
+
+is_tag=false
+git_version=''
+
+# Parse version from build.xml so we can verify version against release tags and use the build.xml version
+# for any branches. Truncate from snapshot suffix if needed.
+buildxml_version=`grep 'property\s*name="base.version"' build.xml |sed -ne 's/.*value="\([^"]*\)".*/\1/p'`
+regx_snapshot="([0-9.]+)-SNAPSHOT$"
+if [[ $buildxml_version =~ $regx_snapshot ]]; then
+   buildxml_version=${BASH_REMATCH[1]}
+fi
+
+if [ "$tag" ]; then
+   is_tag=true
+   # Official release
+   regx_tag="cassandra-(([0-9.]+)(-(alpha|beta|rc)[0-9]+)?)$"
+   # Tentative release
+   regx_tag_tentative="(([0-9.]+)(-(alpha|beta|rc)[0-9]+)?)-tentative$"
+   if [[ $tag =~ $regx_tag ]] || [[ $tag =~ $regx_tag_tentative ]]; then
+      git_version=${BASH_REMATCH[1]}
+   else
+      echo "Error: could not recognize version from tag $tag">&2
+      exit 2
+   fi
+   if [ $buildxml_version != $git_version ]; then
+      echo "Error: build.xml version ($buildxml_version) not matching git tag derived version ($git_version)">&2
+      exit 4
+   fi
+   CASSANDRA_VERSION=$git_version
+   CASSANDRA_REVISION='1'
+else
+   # This could be either trunk or any dev branch or SHA, so we won't be able to get the version
+   # from the branch name. In this case, fall back to version specified in build.xml.
+   CASSANDRA_VERSION="${buildxml_version}"
+   dt=`date +"%Y%m%d"`
+   ref=`git rev-parse --short HEAD`
+   CASSANDRA_REVISION="${dt}git${ref}"
+fi
+
+# Artifact will only be used internally for build process and won't be found with snapshot suffix
+ant artifacts -Drelease=true -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
+cp ${DIST_DIR}/apache-cassandra-*-src.tar.gz ${RPM_BUILD_DIR}/SOURCES/
+
+# if CASSANDRA_VERSION is -alphaN, -betaN, -rcN, then rpmbuild fails on the '-' char; replace with '~'
+CASSANDRA_VERSION=${CASSANDRA_VERSION/-/\~}
+
+command -v python >/dev/null 2>&1 || alias python=/usr/bin/python3
+rpmbuild --define="version ${CASSANDRA_VERSION}" --define="revision ${CASSANDRA_REVISION}" --define="_topdir ${RPM_BUILD_DIR}" -ba ${RPM_SPEC}
+cp ${RPM_BUILD_DIR}/SRPMS/*.rpm ${RPM_BUILD_DIR}/RPMS/noarch/*.rpm ${DIST_DIR}
+
+popd >/dev/null

.build/docker/_create_user.sh

@@ -0,0 +1,64 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+################################
+#
+# Prep
+#
+################################
+
+if [ "$1" == "-h" ]; then
+   echo "$0 [-h] <username> <uid> <gid>"
+   echo " this script is used internally by other scripts in the same directory to create a user with the running host user's same uid and gid"
+   exit 1
+fi
+
+# arguments
+username=$1
+uid=$2
+gid=$3
+
+################################
+#
+# Main
+#
+################################
+
+if grep "^ID=" /etc/os-release | grep -q 'debian\|ubuntu' ; then
+   adduser --quiet --disabled-login --no-create-home --uid $uid --gecos ${username} ${username}
+   groupmod --non-unique -g $gid $username
+   gpasswd -a ${username} sudo >/dev/null
+else
+   adduser --no-create-home --uid $uid ${username}
+fi
+
+# sudo priviledges
+echo "${username} ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/${username}
+chmod 0440 /etc/sudoers.d/${username}
+mkdir -p ${BUILD_HOME}/docker ${DIST_DIR} ${BUILD_HOME}/.ssh
+
+# we need to make SSH less strict to prevent various dtests from failing when they attempt to
+# git clone a given commit/tag/etc
+echo 'Host *\n UserKnownHostsFile /dev/null\n StrictHostKeyChecking no' > ${BUILD_HOME}/.ssh/config
+
+# proper permissions
+chown ${username}:${username} ${BUILD_HOME} ${BUILD_HOME}/docker ${DIST_DIR} ${BUILD_HOME}/.ssh ${BUILD_HOME}/.ssh/config
+chmod og+wx ${BUILD_HOME} ${DIST_DIR}
+chmod 600 ${BUILD_HOME}/.ssh/config
+
+# disable git directory ownership checks
+su ${username} -c "git config --global safe.directory '*'"

.build/docker/_docker_init_tests.sh

@@ -0,0 +1,39 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# pre-conditions
+[ "x" != "x${DIST_DIR}" ] || { echo "DIST_DIR must be defined"; exit 1 ; }
+[ "x" != "x${TEST_SCRIPT}" ] || { echo "TEST_SCRIPT must be defined"; exit 1 ; }
+[ "x" != "x${CASSANDRA_DIR}" ] || { echo "CASSANDRA_DIR must be defined"; exit 1 ; }
+
+# usage
+if [ "$1" == "-h" ]; then
+   echo "$0 [-h] ..."
+   echo " this script is used by run-tests.sh (in the same directory) as a wrapper delegating the execution of the ${TEST_SCRIPT}. all arguments are passed through as-is to ${TEST_SCRIPT}"
+   exit 1
+fi
+
+pushd "${CASSANDRA_DIR}" >/dev/null
+
+echo "Running ${TEST_SCRIPT} $@"
+.build/${TEST_SCRIPT} "$@"
+status=$?
+if [ -d "${DIST_DIR}/test/logs" ]; then
+    find "${DIST_DIR}/test/logs" -type f -name "*.log" | xargs xz -qq
+fi
+popd >/dev/null
+exit ${status}

.build/docker/_docker_run.sh

@@ -0,0 +1,125 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# Creates the artifacts, performing additional QA checks
+#
+# Usage: _docker_run.sh <docker_image_name> <script_to_execute> <java_version>
+
+################################
+#
+# Prep
+#
+################################
+
+# variables, with defaults
+[ "x${cassandra_dir}" != "x" ] || cassandra_dir="$(readlink -f $(dirname "$0")/../..)"
+[ "x${build_dir}" != "x" ] || build_dir="${cassandra_dir}/build"
+[ -d "${build_dir}" ] || { mkdir -p "${build_dir}" ; }
+
+java_version_default=`grep 'property\s*name="java.default"' ${cassandra_dir}/build.xml |sed -ne 's/.*value="\([^"]*\)".*/\1/p'`
+java_version_supported=`grep 'property\s*name="java.supported"' ${cassandra_dir}/build.xml |sed -ne 's/.*value="\([^"]*\)".*/\1/p'`
+
+if [ "$1" == "-h" ]; then
+   echo "$0 [-h] <dockerfile> <run_script> [<java version>]"
+   echo " this script is used by check|build*.sh scripts (in the same directory) as a wrapper delegating the container run of the <dockerfile> and execution of the <run_script>, and using [<java version>] is specified"
+   exit 1
+fi
+
+# arguments
+dockerfile=$1
+run_script=$2
+java_version=$3
+
+# pre-conditions
+command -v docker >/dev/null 2>&1 || { echo >&2 "docker needs to be installed"; exit 1; }
+command -v timeout >/dev/null 2>&1 || { echo >&2 "timeout needs to be installed"; exit 1; }
+(docker info >/dev/null 2>&1) || { echo >&2 "docker needs to running"; exit 1; }
+[ -f "${cassandra_dir}/build.xml" ] || { echo >&2 "${cassandra_dir}/build.xml must exist"; exit 1; }
+[ -f "${cassandra_dir}/.build/docker/${dockerfile}" ] || { echo >&2 "${cassandra_dir}/.build/docker/${dockerfile} must exist"; exit 1; }
+[ -f "${cassandra_dir}/.build/${run_script}" ] || { echo >&2 "${cassandra_dir}/.build/${run_script} must exist"; exit 1; }
+[ "${build_dir:0:1}" == "/" ] || { echo >&2 "\$build_dir must be provided as an absolute path, was ${build_dir}"; exit 1; }
+
+if [ "x${java_version}" == "x" ] ; then
+    echo "Defaulting to java ${java_version_default}"
+    java_version="${java_version_default}"
+fi
+
+regx_java_version="(${java_version_supported//,/|})"
+if [[ ! "${java_version}" =~ $regx_java_version ]]; then
+   echo "Error: Java version is not in ${java_version_supported}, it is set to ${java_version}"
+   exit 1
+fi
+
+# print debug information on versions
+docker --version
+
+# make sure build_dir is good
+chmod -R ag+rwx ${build_dir}
+
+
+################################
+#
+# Main
+#
+################################
+
+pushd ${cassandra_dir}/.build >/dev/null
+
+image_tag="$(md5sum docker/${dockerfile} | cut -d' ' -f1)"
+image_name="apache/cassandra-${dockerfile/.docker/}:${image_tag}"
+
+# Look for existing docker image, otherwise build
+timeout -k 5 5 docker login >/dev/null
+if ! ( [[ "$(docker images -q ${image_name} 2>/dev/null)" != "" ]] || docker pull -q ${image_name} ) ; then
+    # Create build images containing the build tool-chain, Java and an Apache Cassandra git working directory, with retry
+    until docker build -t ${image_name} -f docker/${dockerfile} .  ; do
+        echo "docker build failed… trying again in 10s… "
+        sleep 10
+    done
+fi
+
+# Run build script through docker
+random_string="$(LC_ALL=C tr -dc A-Za-z0-9 </dev/urandom | head -c 6 ; echo '')"
+run_script_name=$(echo ${run_script} | sed  's/.sh//' | sed 's/_//')
+container_name="cassandra_${dockerfile/.docker/}_${un_script_name}_jdk${java_version}__${random_string}"
+
+# Docker commands:
+#  change ant's build directory to $DIST_DIR
+#  set java to java_version
+#  execute the run_script
+docker_command="export ANT_OPTS=\"-Dbuild.dir=\${DIST_DIR} ${CASSANDRA_DOCKER_ANT_OPTS}\" ; \
+                source \${CASSANDRA_DIR}/.build/docker/_set_java.sh ${java_version} ; \
+                \${CASSANDRA_DIR}/.build/${run_script} ${@:4} ; exit \$? "
+
+# run without the default seccomp profile
+# re-use the host's maven repository
+container_id=$(docker run --name ${container_name} -d --security-opt seccomp=unconfined --rm \
+    -v "${cassandra_dir}":/home/build/cassandra -v ~/.m2/repository/:/home/build/.m2/repository/ -v "${build_dir}":/dist \
+    ${build_volume_opt} \
+    ${image_name} sleep 1h)
+
+echo "Running container ${container_name} ${container_id}"
+
+docker exec --user root ${container_name} bash -c "\${CASSANDRA_DIR}/.build/docker/_create_user.sh build $(id -u) $(id -g)"
+docker exec --user build ${container_name} bash -c "${docker_command}"
+RETURN=$?
+
+docker stop ${container_name} >/dev/null
+popd >/dev/null
+[ $RETURN -eq 0 ] && echo "Build directory found at ${build_dir}"
+exit $RETURN

.build/docker/_set_java.sh

@@ -0,0 +1,75 @@
+#!/bin/bash -e
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+################################
+#
+# Prep
+#
+################################
+
+# variables, with defaults
+[ "x${CASSANDRA_DIR}" != "x" ] || { CASSANDRA_DIR="$(pwd)"; }
+[ -d "${CASSANDRA_DIR}" ] || { echo >&2 "Directory ${CASSANDRA_DIR} must exist"; exit 1; }
+
+# pre-conditions
+[ -f "${CASSANDRA_DIR}/build.xml" ] || { echo >&2 "${CASSANDRA_DIR}/build.xml must exist"; exit 1; }
+
+java_version_default=`grep 'property\s*name="java.default"' ${CASSANDRA_DIR}/build.xml |sed -ne 's/.*value="\([^"]*\)".*/\1/p'`
+java_version_supported=`grep 'property\s*name="java.supported"' ${CASSANDRA_DIR}/build.xml |sed -ne 's/.*value="\([^"]*\)".*/\1/p'`
+
+if [ "$1" == "-h" ]; then
+   echo "$0 [-h] [<java version>]"
+   echo " if Java version is not set, it is set to ${java_version_default} by default, valid ${java_version_supported}"
+   echo
+   echo " this script is used internally by other scripts in the same directory to ensure the correct java version is used inside the docker container"
+   exit 1
+fi
+
+# arguments
+java_version=$1
+
+[ "x${java_version}" != "x" ] || java_version="${java_version_default}"
+regx_java_version="(${java_version_supported//,/|})"
+if [[ ! "$java_version" =~ $regx_java_version ]]; then
+   echo "Error: Java version is not in ${java_version_supported}, it is set to $java_version"
+   exit 1
+fi
+
+# TODO – remove after CASSANDRA-18255
+if [ "$java_version" == "1.8" ]; then
+   echo "Error: Java 8 is not supported, ref CASSANDRA-18255"
+   exit 1
+fi
+
+################################
+#
+# Main
+#
+################################
+
+if grep "^ID=" /etc/os-release | grep -q 'debian\|ubuntu' ; then
+    sudo update-java-alternatives --set java-1.${java_version}.0-openjdk-$(dpkg --print-architecture)
+else
+    sudo alternatives --set java $(alternatives --display java | grep "family java-${java_version}-openjdk" | cut -d' ' -f1)
+    sudo alternatives --set javac $(alternatives --display javac | grep "family java-${java_version}-openjdk" | cut -d' ' -f1)
+fi
+export JAVA_HOME=$(readlink -f /usr/bin/javac | sed "s:/bin/javac::")
+echo "Cassandra will be built with Java ${java_version}"
+
+# print debug information on versions
+java -version 2>&1
+javac -version 2>&1

.build/docker/almalinux-build.docker

@@ -0,0 +1,60 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM almalinux:8
+MAINTAINER Apache Cassandra <dev@cassandra.apache.org>
+
+# CONTEXT is expected to be cassandra/.build
+
+ENV BUILD_HOME=/home/build
+ENV RPM_BUILD_DIR=$BUILD_HOME/rpmbuild
+ENV DIST_DIR=/dist
+ENV CASSANDRA_DIR=$BUILD_HOME/cassandra
+ARG UID_ARG=1000
+ARG GID_ARG=1000
+
+LABEL org.cassandra.buildenv=almalinux
+
+VOLUME ${DIST_DIR}
+VOLUME ${CASSANDRA_DIR}
+
+RUN echo "Building with arguments:" \
+    && echo " - DIST_DIR=${DIST_DIR}" \
+    && echo " - BUILD_HOME=${BUILD_HOME}" \
+    && echo " - RPM_BUILD_DIR=${RPM_BUILD_DIR}" \
+    && echo " - CASSANDRA_DIR=${CASSANDRA_DIR}" \
+    && echo " - UID_ARG=${UID_ARG}" \
+    && echo " - GID_ARG=${GID_ARG}"
+
+# install deps
+RUN yum -y install \
+   ant \
+   git \
+   java-11-openjdk-devel \
+   java-17-openjdk-devel \
+   make \
+   rpm-build \
+   sudo \
+   python3-pip \
+   procps \
+   rsync
+
+RUN until curl -f -S -s --retry 9 --retry-connrefused --retry-delay 1 http://mirror.centos.org/centos/7/os/x86_64/Packages/ant-junit-1.9.4-2.el7.noarch.rpm -o ant-junit-1.9.4-2.el7.noarch.rpm ; do echo "curl failed… trying again in 10s… " ; sleep 10 ; done
+
+RUN rpm -i --nodeps ant-junit-1.9.4-2.el7.noarch.rpm
+
+# python3 is needed for the gen-doc target
+RUN pip3 install --upgrade pip

.build/docker/build-artifacts.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# Creates the tarball artifact
+
+$(dirname "$0")/_docker_run.sh bullseye-build.docker build-artifacts.sh $1
+exit $?

.build/docker/build-debian.sh

@@ -0,0 +1,35 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+if [ "$1" == "-h" ]; then
+   echo "$0 [-h] [<java_version>]"
+   echo " build debian packages"
+   exit 1
+fi
+
+echo
+echo "==="
+echo "WARNING: this script modifies local versioned files"
+echo "==="
+echo
+
+#
+# Creates the debian package
+
+$(dirname "$0")/_docker_run.sh bullseye-build.docker docker/_build-debian.sh $1
+exit $?

.build/docker/build-redhat.sh

@@ -0,0 +1,46 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+if [ "$1" == "-h" ]; then
+   echo "$0 [-h] [rpm|noboolean] [<java_version>]"
+   echo " build redhat packages, specify noboolean for centos7 compatibility"
+   exit 1
+fi
+
+# arguments
+rpm_dist=$1
+java_version=$2
+
+if [ "${rpm_dist}" == "noboolean" ]; then
+    dist_name="centos7"
+else # noboolean
+    dist_name="almalinux"
+fi
+
+
+echo
+echo "==="
+echo "WARNING: this script modifies local versioned files"
+echo "==="
+echo
+
+#
+# Creates the redhat package
+
+$(dirname "$0")/_docker_run.sh ${dist_name}-build.docker docker/_build-redhat.sh "${java_version}" ${rpm_dist}
+exit $?

.build/docker/bullseye-build.docker

@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM debian:bullseye
+MAINTAINER Apache Cassandra <dev@cassandra.apache.org>
+
+# CONTEXT is expected to be cassandra/.build
+
+ENV DIST_DIR=/dist
+ENV BUILD_HOME=/home/build
+ENV CASSANDRA_DIR=$BUILD_HOME/cassandra
+
+LABEL org.cassandra.buildenv=bullseye
+
+VOLUME ${DIST_DIR}
+VOLUME ${CASSANDRA_DIR}
+
+RUN echo "Building with arguments:" \
+    && echo " - DIST_DIR=${DIST_DIR}" \
+    && echo " - BUILD_HOME=${BUILD_HOME}" \
+    && echo " - CASSANDRA_DIR=${CASSANDRA_DIR}"
+
+# configure apt to retry downloads
+RUN echo 'APT::Acquire::Retries "99";' > /etc/apt/apt.conf.d/80-retries
+RUN echo 'Acquire::http::Timeout "60";' > /etc/apt/apt.conf.d/80proxy.conf
+RUN echo 'Acquire::ftp::Timeout "60";' >> /etc/apt/apt.conf.d/80proxy.conf
+
+# install deps
+RUN until apt-get update \
+        && apt-get -y install ant build-essential curl devscripts git sudo \
+                python3-pip rsync procps dh-python quilt bash-completion ; \
+    do echo "apt failed… trying again in 10s… " ; sleep 10 ; done
+
+RUN echo 'deb http://deb.debian.org/debian sid main' >> /etc/apt/sources.list
+
+RUN until apt-get update \
+        && apt-get install -y --no-install-recommends openjdk-11-jdk openjdk-17-jdk ; \
+    do echo "apt failed… trying again in 10s… " ; sleep 10 ; done
+
+RUN update-java-alternatives --set java-1.11.0-openjdk-$(dpkg --print-architecture)
+
+# python3 is needed for the gen-doc target
+RUN pip install --upgrade pip

.build/docker/centos7-build.docker

@@ -0,0 +1,89 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM centos:7
+MAINTAINER Apache Cassandra <dev@cassandra.apache.org>
+
+# CONTEXT is expected to be cassandra/.build
+
+ENV BUILD_HOME=/home/build
+ENV RPM_BUILD_DIR=$BUILD_HOME/rpmbuild
+ENV DIST_DIR=/dist
+ENV CASSANDRA_DIR=$BUILD_HOME/cassandra
+ENV ANT_VERSION=1.10.12
+ARG UID_ARG=1000
+ARG GID_ARG=1000
+
+LABEL org.cassandra.buildenv=centos
+
+VOLUME ${DIST_DIR}
+VOLUME ${CASSANDRA_DIR}
+
+RUN echo "Building with arguments:" \
+    && echo " - DIST_DIR=${DIST_DIR}" \
+    && echo " - BUILD_HOME=${BUILD_HOME}" \
+    && echo " - RPM_BUILD_DIR=${RPM_BUILD_DIR}" \
+    && echo " - CASSANDRA_DIR=${CASSANDRA_DIR}" \
+    && echo " - UID_ARG=${UID_ARG}" \
+    && echo " - GID_ARG=${GID_ARG}"
+
+# install deps
+RUN yum -y install \
+   ant \
+   git \
+   java-11-openjdk-devel \
+   make \
+   rpm-build \
+   sudo \
+   python3-pip \
+   rsync
+
+RUN until curl -f -S -s --retry 9 --retry-delay 1 http://mirror.centos.org/centos/7/os/x86_64/Packages/ant-junit-1.9.4-2.el7.noarch.rpm -o ant-junit-1.9.4-2.el7.noarch.rpm ; do echo "curl failed… trying again in 10s… " ; sleep 10 ; done
+
+RUN rpm -i --nodeps ant-junit-1.9.4-2.el7.noarch.rpm
+
+# try first downloads.a.o and then archive.a.o (as the latter has a 5GB per IP daily limit)
+RUN until curl -f -S -s --retry 9 --retry-delay 1 https://downloads.apache.org/ant/binaries/apache-ant-${ANT_VERSION}-bin.tar.gz -o apache-ant-${ANT_VERSION}-bin.tar.gz || curl -f -S -s --retry 9 --retry-delay 1 http://archive.apache.org/dist/ant/binaries/apache-ant-${ANT_VERSION}-bin.tar.gz -o apache-ant-${ANT_VERSION}-bin.tar.gz ; do echo "curl failed… trying again in 10s… " ; sleep 10 ; done
+
+RUN rm -rf /etc/ant.conf
+RUN tar xvf apache-ant-${ANT_VERSION}-bin.tar.gz -C /opt
+RUN ln -sfn /opt/apache-ant-${ANT_VERSION} /opt/ant
+RUN sh -c 'echo ANT_HOME=/opt/ant >> /etc/environment'
+RUN ln -sfn /opt/ant/bin/ant /usr/bin/ant
+
+# jdk17 latest on x64
+RUN if [ "$(arch)" == "x86_64" ] ; then \
+      until curl -f -S -s --retry 9 --retry-delay 1 https://download.oracle.com/java/17/latest/jdk-17_linux-x64_bin.tar.gz -o openjdk-17_bin.tar.gz ; do \
+        echo "curl failed… trying again in 10s… " ; sleep 10 ; \
+      done \
+     fi
+
+# jdk17 latest on arm
+RUN if [ "$(arch)" == "aarch64" ] ; then \
+      until curl -f -S -s --retry 9 --retry-delay 1 https://download.oracle.com/java/17/latest/jdk-17_linux-aarch64_bin.tar.gz -o openjdk-17_bin.tar.gz ; do \
+        echo "curl failed… trying again in 10s… " ; sleep 10 ; \
+      done \
+    fi
+
+RUN tar xvf openjdk-17_bin.tar.gz
+RUN mv jdk-17* /opt/jdk-17
+
+RUN alternatives --install /usr/bin/java java /opt/jdk-17/bin/java 3 --family java-17-openjdk.$(arch)
+RUN alternatives --install /usr/bin/javac javac /opt/jdk-17/bin/javac 3 --family java-17-openjdk.$(arch)
+
+# python3 is needed for the gen-doc target
+RUN pip3 install --upgrade pip
+

.build/docker/check-code.sh

@@ -0,0 +1,23 @@
+#!/bin/sh
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# Perform code checks
+
+export CASSANDRA_DOCKER_ANT_OPTS="-Ddependency-check.home.base=/tmp"
+
+$(dirname "$0")/_docker_run.sh bullseye-build.docker check-code.sh $1

.build/docker/run-tests.sh

@@ -0,0 +1,223 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+#
+# A wrapper script to run-tests.sh (or dtest-python.sh) in docker.
+#  Can split (or grep) the test list into multiple docker runs, collecting results.
+#
+# Each split chunk may be further parallelised over docker containers based on the host's available cpu and memory (and the test type).
+#  Define env variable DISABLE_INNER_SPLITS to disable inner splitting.
+#
+
+# help
+if [ "$#" -lt 1 ] || [ "$#" -gt 3 ] || [ "$1" == "-h" ]; then
+    echo ""
+    echo "Usage: run-tests.sh target [split_chunk|test_regexp] [java_version]"
+    echo ""
+    echo "        default split_chunk is 1/1"
+    echo "        default java_version is what 'java.default' specifies in build.xml"
+    exit 1
+fi
+
+# variables, with defaults
+[ "x${cassandra_dir}" != "x" ] || cassandra_dir="$(readlink -f $(dirname "$0")/../..)"
+[ "x${cassandra_dtest_dir}" != "x" ] || cassandra_dtest_dir="${cassandra_dir}/../cassandra-dtest"
+[ "x${build_dir}" != "x" ] || build_dir="${cassandra_dir}/build"
+[ -d "${build_dir}" ] || { mkdir -p "${build_dir}" ; }
+
+# pre-conditions
+command -v docker >/dev/null 2>&1 || { echo >&2 "docker needs to be installed"; exit 1; }
+command -v bc >/dev/null 2>&1 || { echo >&2 "bc needs to be installed"; exit 1; }
+command -v timeout >/dev/null 2>&1 || { echo >&2 "timeout needs to be installed"; exit 1; }
+(docker info >/dev/null 2>&1) || { echo >&2 "docker needs to running"; exit 1; }
+[ -f "${cassandra_dir}/build.xml" ] || { echo >&2 "${cassandra_dir}/build.xml must exist"; exit 1; }
+[ -f "${cassandra_dir}/.build/run-tests.sh" ] || { echo >&2 "${cassandra_dir}/.build/run-tests.sh must exist"; exit 1; }
+
+# arguments
+target=$1
+split_chunk="1/1"
+[ "$#" -gt 1 ] && split_chunk=$2
+java_version=$3
+
+test_script="run-tests.sh"
+java_version_default=`grep 'property\s*name="java.default"' ${cassandra_dir}/build.xml |sed -ne 's/.*value="\([^"]*\)".*/\1/p'`
+java_version_supported=`grep 'property\s*name="java.supported"' ${cassandra_dir}/build.xml |sed -ne 's/.*value="\([^"]*\)".*/\1/p'`
+
+if [ "x${java_version}" == "x" ] ; then
+    echo "Defaulting to java ${java_version_default}"
+    java_version="${java_version_default}"
+fi
+
+regx_java_version="(${java_version_supported//,/|})"
+if [[ ! "${java_version}" =~ $regx_java_version ]]; then
+    echo "Error: Java version is not in ${java_version_supported}, it is set to ${java_version}"
+    exit 1
+fi
+
+python_version="3.6"
+command -v python >/dev/null 2>&1 && python_version="$(python -V | awk '{print $2}' | awk -F'.' '{print $1"."$2}')"
+
+# print debug information on versions
+docker --version
+
+pushd ${cassandra_dir}/.build >/dev/null
+
+# build test image
+dockerfile="ubuntu2004_test.docker"
+image_tag="$(md5sum docker/${dockerfile} | cut -d' ' -f1)"
+image_name="apache/cassandra-${dockerfile/.docker/}:${image_tag}"
+docker_mounts="-v ${cassandra_dir}:/home/cassandra/cassandra -v "${build_dir}":/home/cassandra/cassandra/build -v ${HOME}/.m2/repository:/home/cassandra/.m2/repository"
+
+# Look for existing docker image, otherwise build
+timeout -k 5 5 docker login >/dev/null 2>/dev/null
+if ! ( [[ "$(docker images -q ${image_name} 2>/dev/null)" != "" ]] || docker pull -q ${image_name} ) >/dev/null 2>/dev/null ; then
+    # Create build images containing the build tool-chain, Java and an Apache Cassandra git working directory, with retry
+    until docker build -t ${image_name} -f docker/${dockerfile} .  ; do
+        echo "docker build failed… trying again in 10s… "
+        sleep 10
+    done
+fi
+
+pushd ${cassandra_dir} >/dev/null
+
+# Optional lookup of Jenkins environment to see how many executors on this machine. `jenkins_executors=1` is used for anything non-jenkins.
+jenkins_executors=1
+if [[ ! -z ${JENKINS_URL+x} ]] && [[ ! -z ${NODE_NAME+x} ]] ; then
+    fetched_jenkins_executors=$(curl -s --retry 9 --retry-connrefused --retry-delay 1 "${JENKINS_URL}/computer/${NODE_NAME}/api/json?pretty=true" | grep 'numExecutors' | awk -F' : ' '{print $2}' | cut -d',' -f1)
+    # use it if we got a valid number (despite retry settings the curl above can still fail
+    [[ ${fetched_jenkins_executors} =~ '^[0-9]+$' ]] && jenkins_executors=${fetched_jenkins_executors}
+fi
+
+# find host's available cores and mem
+cores=1
+command -v nproc >/dev/null 2>&1 && cores=$(nproc --all)
+mem=1
+# linux
+command -v free >/dev/null 2>&1 && mem=$(free -b | grep Mem: | awk '{print $2}')
+# macos
+sysctl -n hw.memsize >/dev/null 2>&1 && mem=$(sysctl -n hw.memsize)
+
+# figure out resource limits, scripts, and mounts for the test type
+case ${target} in
+    # test-burn doesn't have enough tests in it to split beyond 8, and burn and long we want a bit more resources anyway
+    "stress-test" | "fqltool-test" | "microbench" | "test-burn" | "long-test" | "cqlsh-test" )
+        [[ ${mem} -gt $((5 * 1024 * 1024 * 1024 * ${jenkins_executors})) ]] || { echo >&2 "tests require minimum docker memory 6g (per jenkins executor (${jenkins_executors})), found ${mem}"; exit 1; }
+    ;;
+    "dtest" | "dtest-novnode" | "dtest-offheap" | "dtest-large" | "dtest-large-novnode" | "dtest-upgrade" )
+        [ -f "${cassandra_dtest_dir}/dtest.py" ] || { echo >&2 "${cassandra_dtest_dir}/dtest.py must exist"; exit 1; }
+        [[ ${mem} -gt $((15 * 1024 * 1024 * 1024 * ${jenkins_executors})) ]] || { echo >&2 "dtests require minimum docker memory 16g (per jenkins executor (${jenkins_executors})), found ${mem}"; exit 1; }
+        test_script="run-python-dtests.sh"
+        docker_mounts="${docker_mounts} -v ${cassandra_dtest_dir}:/home/cassandra/cassandra-dtest"
+        # check that ${cassandra_dtest_dir} is valid
+        [ -f "${cassandra_dtest_dir}/dtest.py" ] || { echo >&2 "${cassandra_dtest_dir}/dtest.py not found. please specify 'cassandra_dtest_dir' to point to the local cassandra-dtest source"; exit 1; }
+    ;;
+    "test"| "test-cdc" | "test-compression" | "jvm-dtest" | "jvm-dtest-upgrade")
+        [[ ${mem} -gt $((5 * 1024 * 1024 * 1024 * ${jenkins_executors})) ]] || { echo >&2 "tests require minimum docker memory 6g (per jenkins executor (${jenkins_executors})), found ${mem}"; exit 1; }
+        max_docker_runs_by_cores=$( echo "sqrt( ${cores} / ${jenkins_executors} )" | bc )
+        max_docker_runs_by_mem=$(( ${mem} / ( 5 * 1024 * 1024 * 1024 * ${jenkins_executors} ) ))
+    ;;
+    *)
+    echo "unrecognized \"${target}\""
+    exit 1
+    ;;
+esac
+
+docker_cpus=$(echo "scale=2; ${cores} / ( ${jenkins_executors} )" | bc)
+
+# hack: long-test does not handle limited CPUs
+if [ "${target}" == "long-test" ] ; then
+    docker_flags="-m 5g --memory-swap 5g"
+elif [[ "${target}" =~ dtest* ]] ; then
+    docker_flags="--cpus=${docker_cpus} -m 15g --memory-swap 15g"
+else
+    docker_flags="--cpus=${docker_cpus} -m 5g --memory-swap 5g"
+fi
+docker_flags="${docker_flags} --env-file build/env.list -d --rm"
+
+# make sure build_dir is good
+mkdir -p ${build_dir}/tmp || true
+mkdir -p ${build_dir}/test/logs || true
+mkdir -p ${build_dir}/test/output || true
+chmod -R ag+rwx ${build_dir}
+
+# cython can be used for cqlsh-test
+if [ "$cython" == "yes" ]; then
+    [ "${target}" == "cqlsh-test" ] || { echo "cython is only supported for cqlsh-test"; exit 1; }
+else
+    cython="no"
+fi
+
+# the docker container's env
+touch build/env.list
+cat > build/env.list <<EOF
+TEST_SCRIPT=${test_script}
+JAVA_VERSION=${java_version}
+PYTHON_VERSION=${python_version}
+cython=${cython}
+ANT_OPTS="-Dtesttag.extra=.arch=$(arch).python${python_version}"
+EOF
+
+split_str="0_0"
+if [[ "${split_chunk}" =~ ^[0-9]+/[0-9]+$ ]]; then
+    split_str="${split_chunk/\//_}"
+fi
+
+random_string="$(LC_ALL=C tr -dc A-Za-z0-9 </dev/urandom | head -c 6 ; echo '')"
+
+container_name="cassandra_${dockerfile/.docker/}_${target}_jdk${java_version/./-}_arch-$(arch)_python${python_version/./-}_${split_str}__${random_string}"
+
+logfile="${build_dir}/test/logs/docker_attach_${container_name}.log"
+
+# Docker commands:
+#  set java to java_version
+#  execute the run_script
+docker_command="source \${CASSANDRA_DIR}/.build/docker/_set_java.sh ${java_version} ; \
+            \${CASSANDRA_DIR}/.build/docker/_docker_init_tests.sh ${target} ${split_chunk} ; exit \$?"
+
+# start the container, timeout after 4 hours
+docker_id=$(docker run --name ${container_name} ${docker_flags} ${docker_mounts} ${image_name} sleep 4h)
+
+echo "Running container ${container_name} ${docker_id}"
+
+docker exec --user root ${container_name} bash -c "\${CASSANDRA_DIR}/.build/docker/_create_user.sh cassandra $(id -u) $(id -g)" | tee -a ${logfile}
+docker exec --user root ${container_name} update-alternatives --set python /usr/bin/python${python_version} | tee -a ${logfile}
+
+# capture logs and pid for container
+docker exec --user cassandra ${container_name} bash -c "${docker_command}" | tee -a ${logfile}
+status=$?
+
+if [ "$status" -ne 0 ] ; then
+    echo "${docker_id} failed (${status}), debug…"
+    docker inspect ${docker_id}
+    echo "–––"
+    docker logs ${docker_id}
+    echo "–––"
+    docker ps -a
+    echo "–––"
+    docker info
+    echo "–––"
+    echo "Failure."
+fi
+# docker stop in background, ignore errors
+( nohup docker stop ${docker_id} >/dev/null 2>/dev/null & )
+
+xz -f ${logfile} 2>/dev/null
+
+popd >/dev/null
+popd >/dev/null
+exit ${status}

.build/docker/ubuntu2004_test.docker

@@ -0,0 +1,163 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM ubuntu:20.04
+MAINTAINER Apache Cassandra <dev@cassandra.apache.org>
+
+# CONTEXT is expected to be cassandra/.build
+
+ENV BUILD_HOME=/home/cassandra
+ENV CASSANDRA_DIR=$BUILD_HOME/cassandra
+ENV DIST_DIR=$CASSANDRA_DIR/build
+ENV LANG=en_US.UTF-8
+ENV LC_CTYPE=en_US.UTF-8
+ENV PYTHONIOENCODING=utf-8
+ENV PYTHONUNBUFFERED=true
+
+LABEL org.cassandra.buildenv=ubuntu_2004
+
+VOLUME ${CASSANDRA_DIR}
+VOLUME ${DIST_DIR}
+
+RUN echo "Building with arguments:" \
+    && echo " - DIST_DIR=${DIST_DIR}" \
+    && echo " - BUILD_HOME=${BUILD_HOME}" \
+    && echo " - CASSANDRA_DIR=${CASSANDRA_DIR}" \
+    && echo " - UID_ARG=${UID_ARG}" \
+    && echo " - GID_ARG=${GID_ARG}"
+
+# configure apt to retry downloads
+RUN echo 'APT::Acquire::Retries "99";' > /etc/apt/apt.conf.d/80-retries
+RUN echo 'Acquire::http::Timeout "60";' > /etc/apt/apt.conf.d/80proxy.conf
+RUN echo 'Acquire::ftp::Timeout "60";' >> /etc/apt/apt.conf.d/80proxy.conf
+
+# install our python dependencies and some other stuff we need
+# libev4 libev-dev are for the python driver / libssl-dev is for python3.6
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get install -y --no-install-recommends software-properties-common apt-utils
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    add-apt-repository -y ppa:deadsnakes/ppa && \
+    apt-get update && \
+    apt-get install -y curl git-core python3-pip \
+        python3.6 python3.6-venv python3.6-dev \
+        python3.7 python3.7-venv python3.7-dev \
+        python3.8 python3.8-venv python3.8-dev \
+        python3.11 python3.11-venv python3.11-dev \
+        virtualenv net-tools libev4 libev-dev wget gcc libssl-dev libxml2 libxslt1-dev \
+        vim lsof sudo libjemalloc2 dumb-init locales \
+        openjdk-8-jdk openjdk-11-jdk openjdk-17-jdk ant ant-optional
+
+
+RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.6 2
+RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.7 3
+RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.8 4
+RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.11 5
+RUN python3.6 -m pip install --upgrade pip
+RUN python3.7 -m pip install --upgrade pip
+RUN python3.8 -m pip install --upgrade pip
+
+# generate locales for the standard en_US.UTF8 value we use for testing
+RUN locale-gen en_US.UTF-8
+
+# as we only need the requirements.txt file from the dtest repo, let's just get it from GitHub as a raw asset
+# so we can avoid needing to clone the entire repo just to get this file
+RUN curl https://raw.githubusercontent.com/apache/cassandra-dtest/trunk/requirements.txt --output /opt/requirements.txt
+RUN chmod 0644 /opt/requirements.txt
+
+# now setup python via virtualenv with all of the python dependencies we need according to requirements.txt
+RUN pip3 install virtualenv virtualenv-clone
+RUN pip3 install --upgrade wheel
+
+# make Java 8 the default executable (we use to run all tests against Java 8)
+RUN update-java-alternatives --set java-1.8.0-openjdk-$(dpkg --print-architecture)
+
+# enable legacy TLSv1 and TLSv1.1 (CASSANDRA-16848)
+RUN find /etc -type f -name java.security -exec sed -i 's/TLSv1, TLSv1.1//' {} \;
+RUN find /etc -type f -name java.security -exec sed -i 's/3DES_EDE_CBC$/3DES_EDE_CBC, TLSv1, TLSv1.1/' {} \;
+
+# create and change to cassandra-tmp user, use an rare uid to avoid collision later on
+RUN adduser --disabled-login --uid 901743 --lastuid 901743 --gecos cassandra cassandra-tmp
+RUN gpasswd -a cassandra-tmp sudo
+RUN echo "cassandra-tmp ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/build
+RUN chmod 0440 /etc/sudoers.d/build
+
+# switch to the cassandra user
+USER cassandra-tmp
+ENV HOME ${BUILD_HOME}
+WORKDIR ${BUILD_HOME}
+
+ENV ANT_HOME=/usr/share/ant
+
+# run pip commands and setup virtualenv (note we do this after we switch to cassandra user so we
+# setup the virtualenv for the cassandra user and not the root user by accident) for Python 3.6/3.7/3.8
+# Don't build cython extensions when installing cassandra-driver. During test execution the driver
+# dependency is refreshed via pip install --upgrade, so that driver changes can be pulled in without
+# requiring the image to be rebuilt. Rebuilding compiled extensions is costly and is disabled by
+# default in test jobs using the CASS_DRIVER_X env vars below. However, if the extensions are
+# included in the base image, the compiled objects are not updated by pip at run time, which can
+# cause errors if the tests rely on new driver functionality or bug fixes.
+
+RUN virtualenv --python=python3.6 ${BUILD_HOME}/env3.6
+RUN chmod +x ${BUILD_HOME}/env3.6/bin/activate
+
+RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1 CASS_DRIVER_NO_EXTENSIONS=1 \
+    && source ${BUILD_HOME}/env3.6/bin/activate \
+    && pip3 install --upgrade pip \
+    && pip3 install -r /opt/requirements.txt \
+    && pip3 freeze --user"
+
+RUN virtualenv --python=python3.7 ${BUILD_HOME}/env3.7
+RUN chmod +x ${BUILD_HOME}/env3.7/bin/activate
+
+RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1 CASS_DRIVER_NO_EXTENSIONS=1 \
+    && source ${BUILD_HOME}/env3.7/bin/activate \
+    && pip3 install --upgrade pip \
+    && pip3 install -r /opt/requirements.txt \
+    && pip3 freeze --user"
+
+RUN virtualenv --python=python3.8 ${BUILD_HOME}/env3.8
+RUN chmod +x ${BUILD_HOME}/env3.8/bin/activate
+
+RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1 CASS_DRIVER_NO_EXTENSIONS=1 \
+    && source ${BUILD_HOME}/env3.8/bin/activate \
+    && pip3 install --upgrade pip \
+    && pip3 install -r /opt/requirements.txt \
+    && pip3 freeze --user"
+
+RUN virtualenv --python=python3.11 ${BUILD_HOME}/env3.11
+RUN chmod +x ${BUILD_HOME}/env3.11/bin/activate
+
+RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1 CASS_DRIVER_NO_EXTENSIONS=1 \
+    && source ${BUILD_HOME}/env3.11/bin/activate \
+    && curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11 \
+    && pip3 install -r /opt/requirements.txt \
+    && pip3 freeze --user"
+
+# Initialize the CCM git repo as well as this also can fail to clone
+RUN /bin/bash -c "source ${BUILD_HOME}/env3.6/bin/activate && \
+    ccm create -n 1 -v git:trunk test && ccm remove test && \
+    ccm create -n 1 -v git:cassandra-4.1 test && ccm remove test && \
+    ccm create -n 1 -v git:cassandra-4.0 test && ccm remove test"
+
+# the .git subdirectories to pip installed cassandra-driver breaks virtualenv-clone, so just remove them
+RUN rm -rf ${BUILD_HOME}/env*/src/cassandra-driver/.git
+# fix permissions, runtime user has different uid/gid
+RUN chmod -R og+wx ${BUILD_HOME}/.ccm ${BUILD_HOME}/env* ${BUILD_HOME}/.cache
+
+# mark "/tmp" as a volume so it will get mounted as an ext4 mount and not
+# the stupid aufs/CoW stuff that the actual docker container mounts will have.
+# we've been seeing 3+ minute hangs when calling sync on an aufs backed mount
+# so it greatly makes tests flaky as things can hang basically anywhere
+VOLUME ["/tmp"]

.build/run-python-dtests.sh

@@ -0,0 +1,166 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# Wrapper script for running a split chunk of a pytest run of cassandra-dtest
+#
+# Usage: dtest-python.sh target split_chunk
+#  split_chunk formatted as "K/N" for the Kth chunk of N chunks
+
+################################
+#
+# Prep
+#
+################################
+
+# Pass in target to run, defaults to dtest
+DTEST_TARGET="${1:-dtest}"
+# Optional: pass in chunk to test, formatted as "K/N" for the Kth chunk of N chunks
+DTEST_SPLIT_CHUNK="$2"
+
+# variables, with defaults
+[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)"
+[ "x${CASSANDRA_DTEST_DIR}" != "x" ] || CASSANDRA_DTEST_DIR="${CASSANDRA_DIR}/../cassandra-dtest"
+[ "x${DIST_DIR}" != "x" ] || DIST_DIR="${CASSANDRA_DIR}/build"
+
+export PYTHONIOENCODING="utf-8"
+export PYTHONUNBUFFERED=true
+export CASS_DRIVER_NO_EXTENSIONS=true
+export CASS_DRIVER_NO_CYTHON=true
+export CCM_MAX_HEAP_SIZE="1024M"
+export CCM_HEAP_NEWSIZE="512M"
+export CCM_CONFIG_DIR=${DIST_DIR}/.ccm
+export NUM_TOKENS="16"
+#Have Cassandra skip all fsyncs to improve test performance and reliability
+export CASSANDRA_SKIP_SYNC=true
+export TMPDIR="$(mktemp -d /tmp/run-python-dtest.XXXXXX)"
+
+# pre-conditions
+command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; }
+command -v virtualenv >/dev/null 2>&1 || { echo >&2 "virtualenv needs to be installed"; exit 1; }
+[ -f "${CASSANDRA_DIR}/build.xml" ] || { echo >&2 "${CASSANDRA_DIR}/build.xml must exist"; exit 1; }
+[ -d "${DIST_DIR}" ] || { mkdir -p "${DIST_DIR}" ; }
+
+java_version=$(java -version 2>&1 | awk -F '"' '/version/ {print $2}' | awk -F. '{print $1}')
+version=$(grep 'property\s*name=\"base.version\"' ${CASSANDRA_DIR}/build.xml |sed -ne 's/.*value=\"\([^"]*\)\".*/\1/p')
+java_version_default=`grep 'property\s*name="java.default"' ${CASSANDRA_DIR}/build.xml |sed -ne 's/.*value="\([^"]*\)".*/\1/p'`
+
+if [ "${java_version}" -eq 17 ] && [[ "${target}" == "dtest-upgrade" ]] ; then
+    echo "Invalid JDK${java_version}. Only overlapping supported JDKs can be used when upgrading, as the same jdk must be used over the upgrade path."
+    exit 1
+fi
+
+python_version=$(python -V | awk '{print $2}' | awk -F'.' '{print $1"."$2}')
+
+# check project is already built. no cleaning is done, so jenkins unstash works, beware.
+[[ -f "${DIST_DIR}/apache-cassandra-${version}.jar" ]] || [[ -f "${DIST_DIR}/apache-cassandra-${version}-SNAPSHOT.jar" ]] || { echo "Project must be built first. Use \`ant jar\`. Build directory is ${DIST_DIR} with: $(ls ${DIST_DIR})"; exit 1; }
+
+# check if dist artifacts exist, this breaks the dtests
+[[ -d "${DIST_DIR}/dist" ]] && { echo "dtests don't work when build/dist ("${DIST_DIR}/dist") exists (from \`ant artifacts\`)"; exit 1; }
+
+# print debug information on versions
+java -version
+ant -version
+python --version
+virtualenv --version
+
+# cheap trick to ensure dependency libraries are in place. allows us to stash only project specific build artifacts.
+ant -quiet -silent resolver-dist-lib
+
+# Set up venv with dtest dependencies
+set -e # enable immediate exit if venv setup fails
+
+# fresh virtualenv and test logs results everytime
+rm -rf ${DIST_DIR}/venv ${DIST_DIR}/test/{html,output,logs}
+
+# re-use when possible the pre-installed virtualenv found in the cassandra-ubuntu2004_test docker image
+virtualenv-clone ${BUILD_HOME}/env${python_version} ${DIST_DIR}/venv || virtualenv --python=python${python_version} ${DIST_DIR}/venv
+source ${DIST_DIR}/venv/bin/activate
+pip3 install --exists-action w -r ${CASSANDRA_DTEST_DIR}/requirements.txt
+pip3 freeze
+
+################################
+#
+# Main
+#
+################################
+
+cd ${CASSANDRA_DTEST_DIR}
+
+set +e # disable immediate exit from this point
+if [ "${DTEST_TARGET}" = "dtest" ]; then
+    DTEST_ARGS="--use-vnodes --num-tokens=${NUM_TOKENS} --skip-resource-intensive-tests --keep-failed-test-dir"
+elif [ "${DTEST_TARGET}" = "dtest-novnode" ]; then
+    DTEST_ARGS="--skip-resource-intensive-tests --keep-failed-test-dir"
+elif [ "${DTEST_TARGET}" = "dtest-offheap" ]; then
+    DTEST_ARGS="--use-vnodes --num-tokens=${NUM_TOKENS} --use-off-heap-memtables --skip-resource-intensive-tests --keep-failed-test-dir"
+elif [ "${DTEST_TARGET}" = "dtest-large" ]; then
+    DTEST_ARGS="--use-vnodes --num-tokens=${NUM_TOKENS} --only-resource-intensive-tests --force-resource-intensive-tests --keep-failed-test-dir"
+elif [ "${DTEST_TARGET}" = "dtest-large-novnode" ]; then
+    DTEST_ARGS="--only-resource-intensive-tests --force-resource-intensive-tests --keep-failed-test-dir"
+elif [ "${DTEST_TARGET}" = "dtest-upgrade" ]; then
+    DTEST_ARGS="--execute-upgrade-tests-only --upgrade-target-version-only --upgrade-version-selection all"
+else
+    echo "Unknown dtest target: ${DTEST_TARGET}"
+    exit 1
+fi
+
+touch ${DIST_DIR}/test_list.txt
+./run_dtests.py --cassandra-dir=${CASSANDRA_DIR} ${DTEST_ARGS} --dtest-print-tests-only --dtest-print-tests-output=${DIST_DIR}/test_list.txt 2>&1 > ${DIST_DIR}/test_stdout.txt
+[[ $? -eq 0 ]] || { cat ${DIST_DIR}/test_stdout.txt ; exit 1; }
+
+if [[ "${DTEST_SPLIT_CHUNK}" =~ ^[0-9]+/[0-9]+$ ]]; then
+    split_cmd=split
+    ( split --help 2>&1 ) | grep -q "r/K/N" || split_cmd=gsplit
+    command -v ${split_cmd} >/dev/null 2>&1 || { echo >&2 "${split_cmd} needs to be installed"; exit 1; }
+    SPLIT_TESTS=$(${split_cmd} -n r/${DTEST_SPLIT_CHUNK} ${DIST_DIR}/test_list.txt)
+elif [[ "x" != "x${_split_chunk}" ]] ; then
+    SPLIT_TESTS=$(grep "${DTEST_SPLIT_CHUNK}" ${DIST_DIR}/test_list.txt)
+else
+    SPLIT_TESTS=$(cat ${DIST_DIR}/test_list.txt)
+fi
+
+
+PYTEST_OPTS="-vv --log-cli-level=DEBUG --junit-xml=${DIST_DIR}/test/output/nosetests.xml --junit-prefix=${DTEST_TARGET} -s"
+pytest ${PYTEST_OPTS} --cassandra-dir=${CASSANDRA_DIR} ${DTEST_ARGS} ${SPLIT_TESTS} 2>&1 | tee -a ${DIST_DIR}/test_stdout.txt
+
+# tar up any ccm logs for easy retrieval
+if ls ${TMPDIR}/test/*/logs/* &>/dev/null ; then
+    mkdir -p ${DIST_DIR}/test/logs
+    tar -C ${TMPDIR} -cJf ${DIST_DIR}/test/logs/ccm_logs.tar.xz */test/*/logs/*
+fi
+
+# merge all unit xml files into one, and print summary test numbers
+pushd ${CASSANDRA_DIR}/ >/dev/null
+# remove <testsuites> wrapping elements. `ant generate-unified-test-report` doesn't like it`
+sed -r "s/<[\/]?testsuites>//g" ${DIST_DIR}/test/output/nosetests.xml > /tmp/nosetests.xml
+cat /tmp/nosetests.xml > ${DIST_DIR}/test/output/nosetests.xml
+ant -quiet -silent generate-unified-test-report
+popd  >/dev/null
+
+################################
+#
+# Clean
+#
+################################
+
+rm -rf ${TMPDIR}
+unset TMPDIR
+deactivate
+
+# Exit cleanly for usable "Unstable" status
+exit 0

.build/run-tests.sh

@@ -0,0 +1,232 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# Wrapper script for running a split chunk of an ant test target
+#
+# Usage: run-tests.sh target [split_chunk]
+#  split_chunk formatted as "K/N" for the Kth chunk of N chunks
+
+set -o errexit
+set -o pipefail
+
+# variables, with defaults
+[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)"
+[ "x${DIST_DIR}" != "x" ] || DIST_DIR="${CASSANDRA_DIR}/build"
+
+# pre-conditions
+command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; }
+command -v git >/dev/null 2>&1 || { echo >&2 "git needs to be installed"; exit 1; }
+[ -d "${CASSANDRA_DIR}" ] || { echo >&2 "Directory ${CASSANDRA_DIR} must exist"; exit 1; }
+[ -f "${CASSANDRA_DIR}/build.xml" ] || { echo >&2 "${CASSANDRA_DIR}/build.xml must exist"; exit 1; }
+[ -d "${DIST_DIR}" ] || { mkdir -p "${DIST_DIR}" ; }
+
+
+# help
+if [ "$#" -lt 1 ] || [ "$#" -gt 2 ] || [ "$1" == "-h" ]; then
+    echo ""
+    echo "Usage: run-tests.sh target [split_chunk|test_regexp]"
+    echo ""
+    echo "        default split_chunk is 1/1"
+    exit 1
+fi
+
+# print debug information on versions
+ant -version
+git --version
+java -version  2>&1
+javac -version  2>&1
+
+# lists all tests for the specific test type
+_list_tests() {
+  local -r classlistprefix="$1"
+  find "test/${classlistprefix}" -name '*Test.java' | sed "s;^test/${classlistprefix}/;;g" | sort
+}
+
+_split_tests() {
+  local -r _split_chunk="$1"
+  split_cmd=split
+  if [[ "${_split_chunk}" =~ ^[0-9]+/[0-9]+$ ]]; then
+    ( split --help 2>&1 ) | grep -q "r/K/N" || split_cmd=gsplit
+    command -v ${split_cmd} >/dev/null 2>&1 || { echo >&2 "${split_cmd} needs to be installed"; exit 1; }
+    ${split_cmd} -n r/${_split_chunk}
+  elif [[ "x" != "x${_split_chunk}" ]] ; then
+    grep ${_split_chunk}
+  else
+    echo
+  fi
+}
+
+_timeout_for() {
+  grep "name=\"${1}\"" build.xml | awk -F'"' '{print $4}'
+}
+
+_build_all_dtest_jars() {
+    pushd $TMP_DIR >/dev/null
+    rm -fR ${TMP_DIR}/cassandra-dtest-jars
+    until git clone --quiet --depth 1 --no-single-branch https://github.com/apache/cassandra.git cassandra-dtest-jars ; do echo "git clone failed… trying again… " ; done
+
+    # cassandra-4 branches need CASSANDRA_USE_JDK11 to allow jdk11
+    [ "${java_version}" -eq 11 ] && export CASSANDRA_USE_JDK11=true
+
+    pushd ${TMP_DIR}/cassandra-dtest-jars >/dev/null
+    for branch in cassandra-4.0 cassandra-4.1 trunk ; do
+        git checkout $branch
+        dtest_jar_version=$(grep 'property\s*name=\"base.version\"' build.xml |sed -ne 's/.*value=\"\([^"]*\)\".*/\1/p')
+        if [ -f "${DIST_DIR}/dtest-${dtest_jar_version}.jar" ] ; then
+            echo "Skipping dtest jar build for branch ${branch} as ${DIST_DIR}/dtest-${dtest_jar_version}.jar already exists"
+            continue
+        fi
+        # redefine the build.dir to local build folder, rightmost definition wins with java command line system properties
+        ant realclean -Dbuild.dir=${TMP_DIR}/cassandra-dtest-jars/build
+        ant jar dtest-jar ${ANT_TEST_OPTS} -Dbuild.dir=${TMP_DIR}/cassandra-dtest-jars/build
+        cp "${TMP_DIR}/cassandra-dtest-jars/build/dtest-${dtest_jar_version}.jar" ${DIST_DIR}/
+    done
+    popd >/dev/null
+    popd >/dev/null
+    ls -l ${DIST_DIR}/dtest*.jar
+    unset CASSANDRA_USE_JDK11
+}
+
+_run_testlist() {
+    local _target_prefix=$1
+    local _testlist_target=$2
+    local _split_chunk=$3
+    local _test_timeout=$4
+    testlist="$( _list_tests "${_target_prefix}" | _split_tests "${_split_chunk}")"
+    if [[ "${_split_chunk}" =~ ^[0-9]+/[0-9]+$ ]]; then
+      if [[ -z "${testlist}" ]]; then
+        # something has to run in the split to generate a junit xml result
+        echo "Hacking ${_target_prefix} ${_testlist_target} to run only first test found as no tests in split ${_split_chunk} were found"
+        testlist="$( _list_tests "${_target_prefix}" | head -n1)"
+      fi
+    else
+      if [[ -z "${testlist}" ]]; then
+        echo "No tests match ${_split_chunk}"
+        exit 1
+      fi
+    fi
+    ant $_testlist_target -Dtest.classlistprefix="${_target_prefix}" -Dtest.classlistfile=<(echo "${testlist}") -Dtest.timeout="${_test_timeout}" ${ANT_TEST_OPTS} || echo "failed ${_target_prefix} ${_testlist_target}  ${split_chunk}"
+}
+
+_main() {
+  # parameters
+  local -r target="${1:-}"
+
+  local -r split_chunk="${2:-'1/1'}" # Optional: pass in chunk or regexp to test. Chunks formatted as "K/N" for the Kth chunk of N chunks
+  # check split_chunk is compatible with target (if not a regexp)
+  if [[ "${_split_chunk}" =~ ^\d+/\d+$ ]] && [[ "1/1" != "${split_chunk}" ]] ; then
+    case ${target} in
+      "stress-test" | "fqltool-test" | "microbench" | "cqlsh-test")
+          echo "Target ${target} does not suport splits."
+          exit 1
+          ;;
+        *)
+          ;;
+    esac
+  fi
+
+  pushd ${CASSANDRA_DIR}/ >/dev/null
+
+  # jdk check
+  local -r java_version=$(java -version 2>&1 | awk -F '"' '/version/ {print $2}' | awk -F. '{print $1}')
+  local -r version=$(grep 'property\s*name=\"base.version\"' build.xml |sed -ne 's/.*value=\"\([^"]*\)\".*/\1/p')
+  local -r java_version_default=`grep 'property\s*name="java.default"' build.xml |sed -ne 's/.*value="\([^"]*\)".*/\1/p'`
+
+  if [ "${java_version}" -eq 17 ] && [[ "${target}" == "jvm-dtest-upgrade" ]] ; then
+    echo "Invalid JDK${java_version}. Only overlapping supported JDKs can be used when upgrading, as the same jdk must be used over the upgrade path."
+    exit 1
+  fi
+
+  # check project is already built. no cleaning is done, so jenkins unstash works, beware.
+  [[ -f "${DIST_DIR}/apache-cassandra-${version}.jar" ]] || [[ -f "${DIST_DIR}/apache-cassandra-${version}-SNAPSHOT.jar" ]] || { echo "Project must be built first. Use \`ant jar\`. Build directory is ${DIST_DIR} with: $(ls ${DIST_DIR})"; exit 1; }
+
+  # ant test setup
+  export TMP_DIR="${DIST_DIR}/tmp"
+  mkdir -p "${TMP_DIR}" || true
+  export ANT_TEST_OPTS="-Dno-build-test=true -Dtmp.dir=${TMP_DIR} -Drat.skip=true -Dno-checkstyle=true -Dno-javadoc=true -Dant.gen-doc.skip=true"
+
+  # fresh virtualenv and test logs results everytime
+  rm -rf ${DIST_DIR}/test/{html,output,logs}
+
+  # cheap trick to ensure dependency libraries are in place. allows us to stash only project specific build artifacts.
+  ant -quiet -silent resolver-dist-lib
+
+  case ${target} in
+    "stress-test")
+      # hard fail on test compilation, but dont fail the test run as unstable test reports are processed
+      ant stress-build-test ${ANT_TEST_OPTS}
+      ant $target ${ANT_TEST_OPTS} || echo "failed ${target} ${split_chunk}"
+      ;;
+    "fqltool-test")
+      # hard fail on test compilation, but dont fail the test run so unstable test reports are processed
+      ant fqltool-build-test ${ANT_TEST_OPTS}
+      ant $target ${ANT_TEST_OPTS} || echo "failed ${target} ${split_chunk}"
+      ;;
+    "microbench")
+      ant $target ${ANT_TEST_OPTS} -Dmaven.test.failure.ignore=true
+      ;;
+    "test")
+      _run_testlist "unit" "testclasslist" "${split_chunk}" "$(_timeout_for 'test.timeout')"
+      ;;
+    "test-cdc")
+      _run_testlist "unit" "testclasslist-cdc" "${split_chunk}" "$(_timeout_for 'test.timeout')"
+      ;;
+    "test-compression")
+      _run_testlist "unit" "testclasslist-compression" "${split_chunk}" "$(_timeout_for 'test.timeout')"
+      ;;
+    "test-burn")
+      _run_testlist "burn" "testclasslist" "${split_chunk}" "$(_timeout_for 'test.burn.timeout')"
+      ;;
+    "long-test")
+      _run_testlist "long" "testclasslist" "${split_chunk}" "$(_timeout_for 'test.long.timeout')"
+      ;;
+    "jvm-dtest")
+      testlist=$( _list_tests "distributed" | grep -v "upgrade" | _split_tests "${split_chunk}")
+      if [[ -z "$testlist" ]]; then
+          # something has to run in the split to generate a junit xml result
+          echo Hacking jvm-dtest to run only first test found as no tests in split ${split_chunk} were found
+          testlist="$( _list_tests "distributed"  | grep -v "upgrade" | head -n1)"
+      fi
+      ant testclasslist -Dtest.classlistprefix=distributed -Dtest.timeout=$(_timeout_for "test.distributed.timeout") -Dtest.classlistfile=<(echo "${testlist}") ${ANT_TEST_OPTS} || echo "failed ${target} ${split_chunk}"
+      ;;
+    "jvm-dtest-upgrade")
+      _build_all_dtest_jars
+      testlist=$( _list_tests "distributed"  | grep "upgrade" | _split_tests "${split_chunk}")
+      if [[ -z "${testlist}" ]]; then
+          # something has to run in the split to generate a junit xml result
+          echo Hacking jvm-dtest-upgrade to run only first test found as no tests in split ${split_chunk} were found
+          testlist="$( _list_tests "distributed"  | grep "upgrade" | head -n1)"
+      fi
+      ant testclasslist -Dtest.classlistprefix=distributed -Dtest.timeout=$(_timeout_for "test.distributed.timeout") -Dtest.classlistfile=<(echo "${testlist}") ${ANT_TEST_OPTS} || echo "failed ${target} ${split_chunk}"
+      ;;
+    "cqlsh-test")
+      ./pylib/cassandra-cqlsh-tests.sh $(pwd)
+      ;;
+    *)
+      echo "unregconized \"${target}\""
+      exit 1
+      ;;
+  esac
+
+  # merge all unit xml files into one, and print summary test numbers
+  ant -quiet -silent generate-unified-test-report
+
+  popd  >/dev/null
+}
+
+_main "$@"

.circleci/config.yml

@@ -215,6 +215,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -3008,6 +3009,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -8483,6 +8485,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -9857,6 +9860,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:

.circleci/config.yml.FREE

@@ -215,6 +215,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -3008,6 +3009,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -8483,6 +8485,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -9857,6 +9860,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:

.circleci/config.yml.PAID

@@ -215,6 +215,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -3008,6 +3009,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -8483,6 +8485,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -9857,6 +9860,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:

.circleci/config_11_and_17.yml

@@ -231,6 +231,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -581,6 +582,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -7651,6 +7653,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -8631,6 +8634,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:

.circleci/config_11_and_17.yml.FREE

@@ -231,6 +231,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -581,6 +582,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -7651,6 +7653,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -8631,6 +8634,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:

.circleci/config_11_and_17.yml.PAID

@@ -231,6 +231,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -581,6 +582,7 @@ jobs:
           export cython="yes"
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -7651,6 +7653,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:
@@ -8631,6 +8634,7 @@ jobs:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: 15m
     - store_test_results:

.circleci/config_template.yml

@@ -3076,6 +3076,7 @@ commands:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: <<parameters.no_output_timeout>>
     - store_test_results:
@@ -3094,6 +3095,7 @@ commands:
             export cython="yes"
             time mv ~/cassandra /tmp
             cd /tmp/cassandra/
+            ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
             ./pylib/cassandra-cqlsh-tests.sh $(pwd)
           no_output_timeout: <<parameters.no_output_timeout>>
       - store_test_results:

.circleci/config_template_11_and_17.yml

@@ -3042,6 +3042,7 @@ commands:
           export PATH=$JAVA_HOME/bin:$PATH
           time mv ~/cassandra /tmp
           cd /tmp/cassandra/
+          ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
           ./pylib/cassandra-cqlsh-tests.sh $(pwd)
         no_output_timeout: <<parameters.no_output_timeout>>
     - store_test_results:
@@ -3060,6 +3061,7 @@ commands:
             export cython="yes"
             time mv ~/cassandra /tmp
             cd /tmp/cassandra/
+            ant jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true -Djavadoc.skip=true
             ./pylib/cassandra-cqlsh-tests.sh $(pwd)
           no_output_timeout: <<parameters.no_output_timeout>>
       - store_test_results:

README.asc

@@ -13,8 +13,9 @@ Issues should be reported on https://issues.apache.org/jira/projects/CASSANDRA/i
 
 Requirements
 ------------
-. Java >= 1.8 (OpenJDK and Oracle JVMS have been tested)
+- Java: see supported versions in build.xml (search for property "java.supported").
-. Python 3.6+ (for cqlsh)
+- Python: for `cqlsh`, see `bin/cqlsh` (search for function "is_supported_version").
+
 
 Getting started
 ---------------

build.xml

@@ -56,7 +56,7 @@
     <property name="build.src.gen-java" value="${basedir}/src/gen-java"/>
     <property name="build.lib" value="${basedir}/lib"/>
     <property name="build.dir" value="${basedir}/build"/>
-    <property name="build.dir.lib" value="${basedir}/build/lib"/>
+    <property name="build.dir.lib" value="${build.dir}/lib"/>
     <property name="build.test.dir" value="${build.dir}/test"/>
     <property name="build.classes" value="${build.dir}/classes"/>
     <property name="build.classes.main" value="${build.classes}/main" />
@@ -88,6 +88,7 @@
     <property name="test.driver.connection_timeout_ms" value="10000"/>
     <property name="test.driver.read_timeout_ms" value="24000"/>
     <property name="test.jvm.args" value="" />
+    <property name="testtag.extra" value="" />
     <property name="dist.dir" value="${build.dir}/dist"/>
     <property name="tmp.dir" value="${java.io.tmpdir}"/>
 
@@ -480,7 +481,7 @@
     </target>
 
     <!-- create properties file with C version -->
-    <target name="createVersionPropFile" depends="get-git-sha">
+    <target name="_createVersionPropFile" depends="_get-git-sha">
       <taskdef name="propertyfile" classname="org.apache.tools.ant.taskdefs.optional.PropertyFile"/>
       <mkdir dir="${version.properties.dir}"/>
       <propertyfile file="${version.properties.dir}/version.properties">
@@ -498,7 +499,7 @@
         </classpath>
         <jvmarg value="-Dstorage-config=${test.conf}"/>
         <jvmarg value="-Dcassandra.reads.thresholds.coordinator.defensive_checks_enabled=true" /> <!-- enable defensive checks -->
-        <jvmarg value="-javaagent:${build.lib}/jamm-${jamm.version}.jar" />
+        <jvmarg value="-javaagent:${build.dir.lib}/jars/jamm-${jamm.version}.jar" />
         <jvmarg value="-ea"/>
         <jvmarg line="${java-jvmargs}"/>
       </java>
@@ -528,12 +529,12 @@
         </javac>
     </target>
 
-    <target depends="init,gen-cql3-grammar,generate-cql-html,generate-jflex-java,rat-check,get-git-sha"
+    <target depends="init,gen-cql3-grammar,generate-cql-html,generate-jflex-java"
             name="build-project">
         <echo message="${ant.project.name}: ${ant.file}"/>
         <!-- Order matters! -->
         <antcall target="_build_java"/>
-        <antcall target="createVersionPropFile"/>
+        <antcall target="_createVersionPropFile"/>
         <copy todir="${build.classes.main}">
             <fileset dir="${build.src.resources}" />
         </copy>
@@ -591,8 +592,8 @@
     <target name="stress-test-some" depends="maybe-build-test" description="Runs stress tests">
         <testmacro inputdir="${stress.test.src}"
                        timeout="${test.timeout}">
-          <test unless:blank="${test.methods}" name="${test.name}" methods="${test.methods}" outfile="build/test/output/TEST-${test.name}-${test.methods}"/>
+          <test unless:blank="${test.methods}" name="${test.name}" methods="${test.methods}" outfile="${build.test.dir}/output/TEST-${test.name}-${test.methods}"/>
-          <test if:blank="${test.methods}" name="${test.name}" outfile="build/test/output/TEST-${test.name}"/>
+          <test if:blank="${test.methods}" name="${test.name}" outfile="${build.test.dir}/output/TEST-${test.name}"/>
         </testmacro>
     </target>
 
@@ -896,7 +897,7 @@
 
     <!-- creates release tarballs -->
     <target name="artifacts" depends="_artifacts-init,gen-doc,sources-jar"
-            description="Create Cassandra release artifacts">
+            description="Create Cassandra tarball and maven artifacts">
       <tar compression="gzip" longfile="gnu"
         destfile="${build.dir}/${final.name}-bin.tar.gz">
 
@@ -1112,6 +1113,9 @@
     <attribute name="showoutput" default="false"/>
 
     <sequential>
+      <fail message="testing with build.test.dir (${build.test.dir}) not pointing to 'build/test/' will fail, test configurations are hardcoded.">
+        <condition><not><equals arg1="${build.test.dir}" arg2="${basedir}/build/test"/></not></condition>
+      </fail>
       <condition property="additionalagent"
                  value="-javaagent:${build.dir.lib}/jars/jacocoagent.jar=destfile=${jacoco.partialexecfile}"
                  else="">
@@ -1131,7 +1135,7 @@
         <jvmarg value="-Dstorage-config=${test.conf}"/>
         <jvmarg value="-Djava.awt.headless=true"/>
         <!-- Cassandra 3.0+ needs <jvmarg line="... ${additionalagent}" /> here! (not value=) -->
-        <jvmarg line="-javaagent:${build.lib}/jamm-${jamm.version}.jar ${additionalagent}" />
+        <jvmarg line="-javaagent:${build.dir.lib}/jars/jamm-${jamm.version}.jar ${additionalagent}" />
         <jvmarg value="-ea"/>
         <jvmarg value="-Djava.io.tmpdir=${tmp.dir}"/>
         <jvmarg value="-Dcassandra.debugrefcount=true"/>
@@ -1143,7 +1147,7 @@
         -->
         <jvmarg value="-XX:SoftRefLRUPolicyMSPerMB=0" />
         <jvmarg value="-XX:ActiveProcessorCount=${cassandra.test.processorCount}" />
-        <jvmarg value="-XX:HeapDumpPath=build/test" />
+        <jvmarg value="-XX:HeapDumpPath=${build.test.dir}" />
         <jvmarg value="-Dcassandra.test.driver.connection_timeout_ms=${test.driver.connection_timeout_ms}"/>
         <jvmarg value="-Dcassandra.test.driver.read_timeout_ms=${test.driver.read_timeout_ms}"/>
         <jvmarg value="-Dcassandra.memtable_row_overhead_computation_step=100"/>
@@ -1151,7 +1155,7 @@
         <jvmarg value="-Dcassandra.test.sstableformatdevelopment=true"/>
         <!-- The first time SecureRandom initializes can be slow if it blocks on /dev/random -->
         <jvmarg value="-Djava.security.egd=file:/dev/urandom" />
-        <jvmarg value="-Dcassandra.testtag=@{testtag}.jdk${ant.java.version}"/>
+        <jvmarg value="-Dcassandra.testtag=@{testtag}.jdk${ant.java.version}${testtag.extra}"/>
         <jvmarg value="-Dcassandra.keepBriefBrief=${cassandra.keepBriefBrief}" />
         <jvmarg value="-Dcassandra.strict.runtime.checks=true" />
         <jvmarg value="-Dcassandra.reads.thresholds.coordinator.defensive_checks_enabled=true" /> <!-- enable defensive checks -->
@@ -1367,8 +1371,8 @@
       </and>
     </condition>
     <testmacro inputdir="${test.unit.src}" timeout="${test.timeout}">
-      <test if="withMethods" name="${test.name}" methods="${test.methods}" outfile="build/test/output/TEST-${test.name}-${test.methods}"/>
+      <test if="withMethods" name="${test.name}" methods="${test.methods}" outfile="${build.test.dir}/output/TEST-${test.name}-${test.methods}"/>
-      <test if="withoutMethods" name="${test.name}" outfile="build/test/output/TEST-${test.name}"/>
+      <test if="withoutMethods" name="${test.name}" outfile="${build.test.dir}/output/TEST-${test.name}"/>
       <jvmarg value="-Dlegacy-sstable-root=${test.data}/legacy-sstables"/>
       <jvmarg value="-Dinvalid-legacy-sstable-root=${test.data}/invalid-legacy-sstables"/>
       <jvmarg value="-Dcassandra.ring_delay_ms=1000"/>
@@ -1499,7 +1503,7 @@
         <formatter type="brief" usefile="false"/>
         <jvmarg value="-Dstorage-config=${test.conf}"/>
         <jvmarg value="-Djava.awt.headless=true"/>
-        <jvmarg value="-javaagent:${build.lib}/jamm-${jamm.version}.jar" />
+        <jvmarg value="-javaagent:${build.dir.lib}/jars/jamm-${jamm.version}.jar" />
         <jvmarg value="-ea"/>
         <jvmarg value="${jvm_xss}"/>
         <jvmarg value="-Dcassandra.memtable_row_overhead_computation_step=100"/>
@@ -1545,7 +1549,7 @@
         <formatter type="brief" usefile="false"/>
         <jvmarg value="-Dstorage-config=${test.conf}"/>
         <jvmarg value="-Djava.awt.headless=true"/>
-        <jvmarg value="-javaagent:${build.lib}/jamm-${jamm.version}.jar" />
+        <jvmarg value="-javaagent:${build.dir.lib}/jars/jamm-${jamm.version}.jar" />
         <jvmarg value="-ea"/>
         <jvmarg value="${jvm_xss}"/>
         <jvmarg value="-Dcassandra.test.use_prepared=${cassandra.test.use_prepared}"/>
@@ -1787,8 +1791,8 @@
     -->
   <target name="test-jvm-dtest-some" depends="maybe-build-test" description="Execute some in-jvm dtests">
     <testmacro inputdir="${test.distributed.src}" timeout="${test.distributed.timeout}" forkmode="once" showoutput="true">
-      <test unless:blank="${test.methods}" name="${test.name}" methods="${test.methods}" outfile="build/test/output/TEST-${test.name}-${test.methods}"/>
+      <test unless:blank="${test.methods}" name="${test.name}" methods="${test.methods}" outfile="${build.test.dir}/output/TEST-${test.name}-${test.methods}"/>
-      <test if:blank="${test.methods}" name="${test.name}" outfile="build/test/output/TEST-${test.name}"/>
+      <test if:blank="${test.methods}" name="${test.name}" outfile="${build.test.dir}/output/TEST-${test.name}"/>
       <jvmarg value="-Dlogback.configurationFile=test/conf/logback-dtest.xml"/>
       <jvmarg value="-Dcassandra.ring_delay_ms=10000"/>
       <jvmarg value="-Dcassandra.tolerate_sstable_size=true"/>
@@ -1796,6 +1800,31 @@
     </testmacro>
   </target>
 
+  <target name="generate-unified-test-report" description="Merge all unit xml files into one, generate html pages, and print summary test numbers">
+      <junitreport todir="${build.dir}">
+          <fileset dir="${build.test.dir}/output">
+              <include name="**/TEST*.xml"/>
+              <include name="**/cqlshlib.xml"/>
+              <include name="**/nosetests.xml"/>
+          </fileset>
+          <!-- FIXME this can easily OOM, need a workaround-->
+          <report todir="${build.test.dir}/html" />
+      </junitreport>
+      <!-- concat the report through a filter chain to extract what you want -->
+      <concat>
+          <fileset file="${build.test.dir}/html/overview-summary.html" />
+          <filterchain>
+              <linecontainsregexp>
+                  <regexp pattern='title="Display all tests"' />
+              </linecontainsregexp>
+              <tokenfilter>
+                  <!-- escaped values of < and > are "&gt;" and "&lt;" -->
+                  <replaceregex pattern='.*all tests.*&gt;(\d+)&lt;.*all failures.*&gt;(\d+)&lt;.*all errors.*&gt;(\d+)&lt;.*all skipped test.*&gt;(\d+)&lt;.*$' replace="[Test Summary] Run: \1, Failed: \2, Errors: \3, Skipped: \4" />
+              </tokenfilter>
+          </filterchain>
+      </concat>
+  </target>
+
   <!-- run microbenchmarks suite -->
   <target name="microbench" depends="build-jmh">
       <java classname="org.openjdk.jmh.Main"
@@ -1868,7 +1897,7 @@
   </target>
 
   <!-- Generate IDEA project description files -->
-  <target name="generate-idea-files" depends="init,resolver-dist-lib,gen-cql3-grammar,generate-jflex-java,createVersionPropFile" description="Generate IDEA files">
+  <target name="generate-idea-files" depends="init,resolver-dist-lib,gen-cql3-grammar,generate-jflex-java,_createVersionPropFile" description="Generate IDEA files">
     <delete dir=".idea"/>
     <delete file="${eclipse.project.name}.iml"/>
     <mkdir dir=".idea"/>
@@ -1919,10 +1948,10 @@
         <fileset dir="lib">
             <include name="**/*.jar" />
         </fileset>
-        <fileset dir="build/lib/jars">
+        <fileset dir="${build.dir.lib}/jars">
             <include name="**/*.jar" />
         </fileset>
-        <fileset dir="build/test/lib/jars">
+        <fileset dir="${test.lib}/jars">
             <include name="**/*.jar" />
         </fileset>
     </path>
@@ -1938,12 +1967,12 @@
   <classpathentry kind="src" path="src/resources"/>
   <classpathentry kind="src" path="src/gen-java"/>
   <classpathentry kind="src" path="conf" including="hotspot_compiler"/>
-  <classpathentry kind="src" output="build/test/classes" path="test/unit"/>
+  <classpathentry kind="src" output="${test.classes}" path="test/unit"/>
-  <classpathentry kind="src" output="build/test/classes" path="test/long"/>
+  <classpathentry kind="src" output="${test.classes}" path="test/long"/>
-  <classpathentry kind="src" output="build/test/classes" path="test/distributed"/>
+  <classpathentry kind="src" output="${test.classes}" path="test/distributed"/>
-  <classpathentry kind="src" output="build/test/classes" path="test/simulator/asm"/>
+  <classpathentry kind="src" output="${test.classes}" path="test/simulator/asm"/>
-  <classpathentry kind="src" output="build/test/classes" path="test/simulator/main"/>
+  <classpathentry kind="src" output="${test.classes}" path="test/simulator/main"/>
-  <classpathentry kind="src" output="build/test/classes" path="test/resources" />
+  <classpathentry kind="src" output="${test.classes}" path="test/resources" />
   <classpathentry kind="src" path="tools/stress/src"/>
   <classpathentry kind="src" path="tools/fqltool/src"/>
   <classpathentry kind="src" output="build/test/stress-classes" path="tools/stress/test/unit" />
@@ -1980,7 +2009,7 @@
   </target>
 
   <!-- ECJ 4.6.1 in standalone mode does not work with JPMS, so we skip this target for Java 11 -->
-  <target name="eclipse-warnings" depends="build, _assert_rat_output" description="Run eclipse compiler code analysis" if="java.version.8">
+  <target name="eclipse-warnings" depends="build" description="Run eclipse compiler code analysis" if="java.version.8">
         <property name="ecj.log.dir" value="${build.dir}/ecj" />
         <property name="ecj.warnings.file" value="${ecj.log.dir}/eclipse_compiler_checks.txt"/>
         <mkdir  dir="${ecj.log.dir}" />

debian/rules

@@ -7,16 +7,12 @@ include /usr/share/quilt/quilt.make
 
 ANT = /usr/bin/ant
 VERSION = $(shell dpkg-parsechangelog | sed -ne 's/^Version: \([^-|~|+]*\).*/\1/p')
-
+BUILD_DIR ?= "build"
-test:
-	dh_testdir
-	$(ANT) test
 
 clean: unpatch
 	dh_clean build-stamp
 	dh_testdir
 	dh_testroot
-	$(ANT) realclean
 	rm -f build-stamp build.properties
 	rm -rf bin/java
 	find -name "*.pyc" -exec rm '{}' ';'
@@ -30,7 +26,7 @@ build-stamp: $(QUILT_STAMPFN)
 	printf "version=%s" $(VERSION) > build.properties
 
 	$(ANT) generate-cql-html
-	$(ANT) jar
+	$(ANT) jar -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true
 	cd pylib && python3 setup.py install --no-compile --install-layout deb \
 		--root $(CURDIR)/debian/cassandra
 
@@ -43,15 +39,15 @@ install: build
 	dh_install
 
 	# Copy in the jar and symlink to something stable
-	dh_install build/apache-cassandra-$(VERSION).jar \
+	dh_install $(BUILD_DIR)/apache-cassandra-$(VERSION).jar \
 		usr/share/cassandra
 
 	# Copy stress jars
-	dh_install build/tools/lib/stress.jar \
+	dh_install $(BUILD_DIR)/tools/lib/stress.jar \
 		usr/share/cassandra
 
 	# Copy fqltool jars
-	dh_install build/tools/lib/fqltool.jar \
+	dh_install $(BUILD_DIR)/tools/lib/fqltool.jar \
 		usr/share/cassandra
 
 	dh_link usr/share/cassandra/apache-cassandra-$(VERSION).jar \

pylib/cassandra-cqlsh-tests.sh

@@ -1,4 +1,4 @@
-#!/bin/bash -x
+#!/bin/bash
 #
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
@@ -25,12 +25,8 @@
 
 WORKSPACE=$1
 
-if [ "${WORKSPACE}" = "" ]; then
+[ "x${WORKSPACE}" != "x" ] || WORKSPACE="$(readlink -f $(dirname "$0")/..)"
-    echo "Specify Cassandra source directory"
+[ "x${BUILD_DIR}" != "x" ] || BUILD_DIR="${WORKSPACE}/build"
-    exit
-fi
-
-PYTHON_VERSION=python3
 
 export PYTHONIOENCODING="utf-8"
 export PYTHONUNBUFFERED=true
@@ -38,43 +34,43 @@ export CASS_DRIVER_NO_EXTENSIONS=true
 export CASS_DRIVER_NO_CYTHON=true
 export CCM_MAX_HEAP_SIZE="2048M"
 export CCM_HEAP_NEWSIZE="200M"
-export CCM_CONFIG_DIR=${WORKSPACE}/.ccm
+export CCM_CONFIG_DIR=${BUILD_DIR}/.ccm
 export NUM_TOKENS="16"
 export CASSANDRA_DIR=${WORKSPACE}
+export TMPDIR="$(mktemp -d /tmp/run-python-dtest.XXXXXX)"
 
 java_version=$(java -version 2>&1 | awk -F '"' '/version/ {print $2}' | awk -F. '{print $1}')
-export TESTSUITE_NAME="cqlshlib.${PYTHON_VERSION}.jdk${java_version}"
+version=$(grep 'property\s*name=\"base.version\"' ${CASSANDRA_DIR}/build.xml |sed -ne 's/.*value=\"\([^"]*\)\".*/\1/p')
 
-ant -buildfile ${CASSANDRA_DIR}/build.xml realclean
+python_version="3.6"
-# Loop to prevent failure due to maven-ant-tasks not downloading a jar..
+command -v python3 >/dev/null 2>&1 && python_version="$(python3 -V | awk '{print $2}' | awk -F'.' '{print $1"."$2}')"
-for x in $(seq 1 3); do
+
-    ant -buildfile ${CASSANDRA_DIR}/build.xml jar
+export TESTSUITE_NAME="cqlshlib.python${python_version}.jdk${java_version}"
-    RETURN="$?"
+
-    if [ "${RETURN}" -eq "0" ]; then
+pushd ${CASSANDRA_DIR} >/dev/null
-        break
+
-    fi
+# check project is already built. no cleaning is done, so jenkins unstash works, beware.
-done
+[[ -f "${BUILD_DIR}/apache-cassandra-${version}.jar" ]] || [[ -f "${BUILD_DIR}/apache-cassandra-${version}-SNAPSHOT.jar" ]] || { echo "Project must be built first. Use \`ant jar\`. Build directory is ${BUILD_DIR} with: $(ls ${BUILD_DIR})"; exit 1; }
-# Exit, if we didn't build successfully
-if [ "${RETURN}" -ne "0" ]; then
-    echo "Build failed with exit code: ${RETURN}"
-    exit ${RETURN}
-fi
 
 # Set up venv with dtest dependencies
 set -e # enable immediate exit if venv setup fails
-virtualenv --python=$PYTHON_VERSION venv
-source venv/bin/activate
-# 3.11 needs the newest pip
-curl -sS https://bootstrap.pypa.io/get-pip.py | $PYTHON_VERSION
 
-pip install -r ${CASSANDRA_DIR}/pylib/requirements.txt
+# fresh virtualenv and test logs results everytime
+rm -fr ${DIST_DIR}/venv ${DIST_DIR}/test/{html,output,logs}
+
+# re-use when possible the pre-installed virtualenv found in the cassandra-ubuntu2004_test docker image
+virtualenv-clone ${BUILD_HOME}/env${python_version} ${BUILD_DIR}/venv || virtualenv --python=python3 ${BUILD_DIR}/venv
+source ${BUILD_DIR}/venv/bin/activate
+
+pip install --exists-action w -r ${CASSANDRA_DIR}/pylib/requirements.txt
 pip freeze
 
 if [ "$cython" = "yes" ]; then
     TESTSUITE_NAME="${TESTSUITE_NAME}.cython"
     pip install "Cython>=0.29.15,<3.0"
-    cd pylib/; python setup.py build_ext --inplace
+    pushd pylib >/dev/null
-    cd ${WORKSPACE}
+    python setup.py build_ext --inplace
+    popd >/dev/null
 else
     TESTSUITE_NAME="${TESTSUITE_NAME}.no_cython"
 fi
@@ -108,14 +104,27 @@ esac
 
 ccm start --wait-for-binary-proto
 
-cd ${CASSANDRA_DIR}/pylib/cqlshlib/
+pushd ${CASSANDRA_DIR}/pylib/cqlshlib/ >/dev/null
 
 set +e # disable immediate exit from this point
-pytest --junitxml=${WORKSPACE}/cqlshlib.xml
+pytest --junitxml=${BUILD_DIR}/test/output/cqlshlib.xml
 RETURN="$?"
 
-sed -i "s/testsuite errors=\(\".*\"\) failures=\(\".*\"\) hostname=\(\".*\"\) name=\"pytest\"/testsuite errors=\1 failures=\2 hostname=\3 name=\"${TESTSUITE_NAME}\"/g" ${WORKSPACE}/cqlshlib.xml
+# remove <testsuites> wrapping elements. `ant generate-unified-test-report` doesn't like it`
-sed -i "s/testcase classname=\"cqlshlib./testcase classname=\"${TESTSUITE_NAME}./g" ${WORKSPACE}/cqlshlib.xml
+sed -r "s/<[\/]?testsuites>//g" ${BUILD_DIR}/test/output/cqlshlib.xml > /tmp/cqlshlib.xml
+cat /tmp/cqlshlib.xml > ${BUILD_DIR}/test/output/cqlshlib.xml
+
+# don't do inline sed for linux+mac compat
+sed "s/testsuite errors=\(\".*\"\) failures=\(\".*\"\) hostname=\(\".*\"\) name=\"pytest\"/testsuite errors=\1 failures=\2 hostname=\3 name=\"${TESTSUITE_NAME}\"/g" ${BUILD_DIR}/test/output/cqlshlib.xml > /tmp/cqlshlib.xml
+cat /tmp/cqlshlib.xml > ${BUILD_DIR}/test/output/cqlshlib.xml
+sed "s/testcase classname=\"cqlshlib./testcase classname=\"${TESTSUITE_NAME}./g" ${BUILD_DIR}/test/output/cqlshlib.xml > /tmp/cqlshlib.xml
+cat /tmp/cqlshlib.xml > ${BUILD_DIR}/test/output/cqlshlib.xml
+
+# tar up any ccm logs for easy retrieval
+if ls ${TMPDIR}/test/*/logs/* &>/dev/null ; then
+    mkdir -p ${DIST_DIR}/test/logs
+    tar -C ${TMPDIR} -cJf ${DIST_DIR}/test/logs/ccm_logs.tar.xz */test/*/logs/*
+fi
 
 ccm remove
 
@@ -125,8 +134,12 @@ ccm remove
 #
 ################################
 
-# /virtualenv
+
+rm -rf ${TMPDIR}
+unset TMPDIR
 deactivate
+popd >/dev/null
+popd >/dev/null
 
 # circleci needs non-zero exit on failures, jenkins need zero exit to process the test failures
 if ! command -v circleci >/dev/null 2>&1

redhat/cassandra.spec

@@ -32,6 +32,9 @@
 %define upstream_version %(echo %{version} | sed -r 's/~/-/g')
 %define relname apache-cassandra-%{upstream_version}
 
+# default DIST_DIR to build
+%global _get_dist_dir %(echo "${DIST_DIR:-build}")
+
 Name:          cassandra
 Version:       %{version}
 Release:       %{revision}
@@ -69,7 +72,7 @@ Cassandra is a distributed (peer-to-peer) system for the management and storage
 %build
 export LANG=en_US.UTF-8
 export JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF-8"
-ant clean jar -Dversion=%{upstream_version}
+ant jar -Dversion=%{upstream_version} -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true
 
 %install
 %{__rm} -rf %{buildroot}
@@ -115,10 +118,10 @@ cp -p redhat/default %{buildroot}/%{_sysconfdir}/default/%{username}
 cp -pr lib/* %{buildroot}/usr/share/%{username}/lib/
 
 # copy stress jar
-cp -p build/tools/lib/stress.jar %{buildroot}/usr/share/%{username}/
+cp -p %{_get_dist_dir}/tools/lib/stress.jar %{buildroot}/usr/share/%{username}/
 
 # copy fqltool jar
-cp -p build/tools/lib/fqltool.jar %{buildroot}/usr/share/%{username}/
+cp -p %{_get_dist_dir}/tools/lib/fqltool.jar %{buildroot}/usr/share/%{username}/
 
 # copy binaries
 mv bin/cassandra %{buildroot}/usr/sbin/
@@ -126,7 +129,7 @@ cp -p bin/* %{buildroot}/usr/bin/
 cp -p tools/bin/* %{buildroot}/usr/bin/
 
 # copy cassandra jar
-cp build/apache-cassandra-%{upstream_version}.jar %{buildroot}/usr/share/%{username}/
+cp %{_get_dist_dir}/apache-cassandra-%{upstream_version}.jar %{buildroot}/usr/share/%{username}/
 
 %clean
 %{__rm} -rf %{buildroot}
@@ -207,6 +210,10 @@ This package contains extra tools for working with Cassandra clusters.
 
 
 %changelog
+# packaging changes, not software changes
+* Thu May 04 2023 Mick Semb Wever <mck@apache.org>
+- 5.0
+- RPM packaging brought in-tree. CASSANDRA-18133
 * Mon Dec 05 2016 Michael Shuler <mshuler@apache.org>
 - 2.1.17, 2.2.9, 3.0.11, 3.10
 - Reintroduce RPM packaging

redhat/noboolean/cassandra.spec

@@ -32,6 +32,9 @@
 %define upstream_version %(echo %{version} | sed -r 's/~/-/g')
 %define relname apache-cassandra-%{upstream_version}
 
+# default DIST_DIR to build
+%global _get_dist_dir %(echo "${DIST_DIR:-build}")
+
 Name:          cassandra
 Version:       %{version}
 Release:       %{revision}
@@ -69,7 +72,7 @@ Cassandra is a distributed (peer-to-peer) system for the management and storage
 %build
 export LANG=en_US.UTF-8
 export JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF-8"
-ant clean jar -Dversion=%{upstream_version}
+ant jar -Dversion=%{upstream_version} -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true
 
 %install
 %{__rm} -rf %{buildroot}
@@ -115,10 +118,10 @@ cp -p redhat/default %{buildroot}/%{_sysconfdir}/default/%{username}
 cp -pr lib/* %{buildroot}/usr/share/%{username}/lib/
 
 # copy stress jar
-cp -p build/tools/lib/stress.jar %{buildroot}/usr/share/%{username}/
+cp -p %{_get_dist_dir}/tools/lib/stress.jar %{buildroot}/usr/share/%{username}/
 
 # copy fqltool jar
-cp -p build/tools/lib/fqltool.jar %{buildroot}/usr/share/%{username}/
+cp -p %{_get_dist_dir}/tools/lib/fqltool.jar %{buildroot}/usr/share/%{username}/
 
 # copy binaries
 mv bin/cassandra %{buildroot}/usr/sbin/
@@ -126,7 +129,7 @@ cp -p bin/* %{buildroot}/usr/bin/
 cp -p tools/bin/* %{buildroot}/usr/bin/
 
 # copy cassandra jar
-cp build/apache-cassandra-%{upstream_version}.jar %{buildroot}/usr/share/%{username}/
+cp %{_get_dist_dir}/apache-cassandra-%{upstream_version}.jar %{buildroot}/usr/share/%{username}/
 
 %clean
 %{__rm} -rf %{buildroot}
@@ -207,6 +210,10 @@ This package contains extra tools for working with Cassandra clusters.
 
 
 %changelog
+# packaging changes, not software changes
+* Thu May 04 2023 Mick Semb Wever <mck@apache.org>
+- 5.0
+- RPM packaging brought in-tree. CASSANDRA-18133
 * Mon Dec 05 2016 Michael Shuler <mshuler@apache.org>
 - 2.1.17, 2.2.9, 3.0.11, 3.10
 - Reintroduce RPM packaging

test/distributed/org/apache/cassandra/distributed/impl/Instance.java

@@ -234,7 +234,7 @@ public class Instance extends IsolatedExecutor implements IInvokableInstance
         if (!f.exists())
             f = new File(FileSystems.getDefault(), String.format("build/test/logs/%s/%s/%s/system.log", tag, clusterId, instanceId));
         if (!f.exists())
-            throw new AssertionError("Unable to locate system.log under " + new File("build/test/logs").absolutePath() + "; make sure ICluster.setup() is called or extend TestBaseImpl and do not define a static beforeClass function with @BeforeClass");
+            throw new AssertionError("Unable to locate system.log under " + f.absolutePath() + "; make sure ICluster.setup() is called or extend TestBaseImpl and do not define a static beforeClass function with @BeforeClass");
         return new FileLogAction(f);
     }