* Add ephemeral-storage requests and limits to agents, and log peak disk usage
Agents were evicted at the node level due to lack of disk space. Each pod needs minimum 100GiB, and each template now budgets 80Gi of the ~89Gi that a 100GiB node allocates, with docker's own volume carries a 60Gi sizeLimit. Exceeding either evicts the pod, so logging has been added in each test split before and after the run. All script blocks in Jenkinsfile are now labeled for improved readability in test runs.
* `.build/run-ci` must not silently drop a site's helm values customisations
Long-lived instances like pre-ci.cassandra.apache.org carry values in addition to the jenkins-deployment.yaml, and any `helm upgrade -f jenkins-deployment.yaml` (or `.build/run-ci`) would drop them. Allow these values to be kept in a separate file, passed in with `--values-override`. Before upgrading helm values, we now compare the deployed values against what is about to be applied and require confirmation for anything only the site holds. This catches added keys and dropped list items (but it cannot catch every change).
Also keep the jenkins-home pvc claim on uninstall, so --only-tear-down no longer destroys build history. Add jenkins-test.sh and a gha workflow for test validating changes under .jenkins/
patch by Mick Semb Wever; reviewed by Dmitry Konstantinov for CASSANDRA-21523
Wires the conventional AGENTS.md -> SECURITY.md -> security model chain so automated tooling
can mechanically discover the project's security model. No model content is changed.
patch by Jarek Potiuk; reviewed by Stefan Miklosovic, Michael Semb Wever for CASSANDRA-21464
Assisted-by: Claude Opus 4.8 (1M context)
Additional fixes
- upgrade ubuntu-test.docker to ubuntu 22.04 (and some small fixes brought forward from CASSANDRA-20997)
- (.jenkins/k8s) changes gke's small node pool from n2-highcpu-8 to e2-highcpu-8 (gke's own pod resources reqs increased)
- (.jenkins/k8s) limit one agent pod per node (when jenkins is deployed in k8s using our helm)
- (.jenkins/k8s) add the cassandra-6.0 job
- pass maven.repo.local sys prop, if set, into accord's gradle build
- check and fail on maven dependencies checksums when downloading
- when on-the-fly building docker images add `--load` (podman compatbility)
- newer python versions can use venv instead of virtualenv
- rename ubuntu2004_test.docker to ubuntu-test.docker (in 5.0)
- (Jenkinsfile) fix retries (which was no longer working when agents/nodes died, since CASSANDRA-20833)
- (Jenkinsfile) @NonCPS where (now) needed
- (Jenkinsfile) fix default value for the branch parameter (after a new helm deploy)
- (.build/run-ci) fix missing parameters on jobs also at helm deploy time
- (.build/run-ci) .git suffixes on repo urls are optional
patch by Mick Semb Wever; reviewed by Dmitry Konstantinov, Stefan Miklosovic for CASSANDRA-21403
Set 1 thread and 1 second for the measurement iteration (note: it is a smoke test only, we do not analyze results from such short measurements)
Exclude some options for heavy benchmarks by default
Shutdown all threads once JMH test is completed to avoid awaiting of a forked JVM till timeout at the end
patch by Dmitry Konstantinov; reviewed by Michael Semb Wever for CASSANDRA-21388
- consumer and producer threads wait signal without acquiring the lock first
- lock owners may prepare more work than they need, moving some dynamically-adjusted portion of the work from the prioritised lock-managed structures onto a non-blocking queue, so that other threads may consume work from there; the portion is continually micro-adjusted to target some available work whenever the lock is acquired.
- adopts/supports some features of the SEPExecutor:
- threads may auto-adjust the number of running threads based on how much time is collectively spent waiting, to minimise time spent signalling
- consumers may (timed-sleep) poll rather than await a signal, reducing the number of kernel interactions needed; relying on the auto-adjustment to bound the time the scheduler spends waking threads with no work to do
Also Fix:
- Client Result should not be persisted or included in Command object at any time
patch by Benedict; reviewed by Alex Petrov and Ariel Weisberg for CASSANDRA-21375
This is the continuation of CASSANDRA-21319 to make branching of cassandra-6.0 done.
The change in .build/run-ci will start to route jobs to cassandra-6.0 pipeline in pre-ci.
patch by Stefan Miklosovic; reviewed by Brandon Williams for CASSANDRA-21319
Includes bug fixes and features:
- Improved observability in AutoRepair (CASSANDRA-20581)
- Stop repair scheduler if two major versions detected (CASSANDRA-20048)
- Safeguard Full repair against disk protection (CASSANDRA-20045)
- Stop AutoRepair monitoring thread upon shutdown (CASSANDRA-20623)
- Fix race condition in auto-repair scheduler (CASSANDRA-20265)
- Minimum repair task duration setting (CASSANDRA-20160)
- Preview_repaired auto-repair type (CASSANDRA-20046)
- Gate auto-repair behind cassandra.autorepair.enable JVM property
- Add cassandra.autorepair.check_min_version to gate minimum version enforcement
- Prevent auto-repair from running if any node is below 5.0.7
- Make system_distributed auto-repair schema conditional on feature being enabled
- Add user-friendly errors for disabled auto-repair and schema incompatibility
patch by Paulo Motta; reviewed by Andy Tolbert, Jaydeepkumar Chovatia for CASSANDRA-21138
Co-Authored-By: Andy Tolbert <andy_tolbert@apple.com>
Co-Authored-By: Chris Lohfink <clohfink@netflix.com>
Co-Authored-By: Francisco Guerrero <frankgh@apache.org>
Co-Authored-By: Himanshu Jindal <himanshj@amazon.com>
Co-Authored-By: Jaydeepkumar Chovatia <jchovati@uber.com>
Co-Authored-By: Kristijonas Zalys <kzalys@uber.com>
Co-Authored-By: jaydeepkumar1984 <chovatia.jaydeep@gmail.com>
mck