As we were implementing the snitch itself, we noticed that the constructors
for cloud-based snitches are unnecessarilly complicated and
we took the opportunity to make them simpler.
patch by Stefan Miklosovic; reviewed by German Eichberger and Jacek Lewandowski for CASSANDRA-18646
Fixes:
- regular expressions when filtering on tests to run, and exit when no match
- safety when removing (cleaning up) directories (${DIST_DIR} and ${TMPDIR})
- pre-condition `ant jar` has been used and not `ant artifacts`
- compatibility with git worktrees (the original git working directory needs to be mounted in its existing path in docker)
- fix tarball for ccm logs
- add `--keep-failed-test-dir` to dtest-upgrade test types
- only do `docker login` if `docker pull` is going to be called
- unset CASSANDRA_HOME when running python dtests
patch by Mick Semb Wever; reviewed by Brandon Williams, Jacek Lewandowski for CASSANDRA-18567, CASSANDRA-18499
The approach is documented in the included UnifiedCompactionStrategy.md.
Also included:
- Generalize prettyPrint to prettyPrintBinary (with 2^10 multiplier,
e.g. MiB) and prettyPrintDecimal (with 1000 multiplier, e.g. ms)
that cover the full range of double, and add a parseHumanReadable
method that can read both.
- CASSANDRA-18123: Fix invalid reuse of metadata collector during flushing
- Fix invalid token range splitting with non-unit weights
- Add flushSizeOnDisk metric
- Add ability to change compaction default in YAML
patch by Alex Sorokoumov, Branimir Lambov, Dimitar Dimitrov and Stefania Alborghetti
reviewed by Alex Sorokoumov, Jaroslaw Grabowski and Maxwell Guo for CASSANDRA-18397
The patch also refactors existing cloud snitches to get rid of the duplicate code,
this is the logical follow-up of CASSANDRA-16555 where AbstractCloudMetadataServiceConnector was introduced.
patch by Stefan Miklosovic; reviewed by Jacek Lewandowski, Jackson Fleming and Maxwell Guo for CASSANDRA-18438
See .build/README.md .Build and test scripts cover all CI activities in an CI agnostic manner. Scripts are found in non-docker and dockerised variants.
The non-docker scripts are found first under .build/ .These interact directly with ant, providing additional checks and environment setup. It is intended that what can be pushed down into the base ant build layer should be. The
debian and redhat build scripts are hidden under .build/docker because they are generally not usuable outside of their corresponding linux dists, and we don't want to encourage anything to the contrary. Bring the artifact/deb/rpm
build scripts (and associated docker images) from cassandra-builds repo to the .build directory. These packaging script for debian and redhat are now separated from the artifacts scripts. An additional build script check-code.sh
has been added that is solely responsible for all linter and code checks (includes owasp dependency checker). Only JDK11 and JDK17 are supported (as JDK8 is soon to be dropped). Scripts do not clean (or realclean) so that
scripts can be better pipelined, but `ant jar` was required to be added to circleci and ci-cassandra.a.o cqlsh-tests. Scripts can define a custom BUILD_DIR (build.dir) which can permit running parrallel builds off the same source
(though not all tests support this, or their support for it has atrophied). All build and test artifacts, results, and logs are found under build/ (or the BUILD_DIR). The build local venv and .ccm directories are now also
created under build (or BUILD_DIR). The test venv, logs and results are wiped each run. Use a separate mktemp directory each run, under /tmp (respect /tmp responsibility, and python fails when tmpdir is on a docker volume).
Don't set CASSANDRA_HOME when running python dtests (and fail-fast if `ant artifacts` was used to build instead of `ant jar`).
The docker scripts are found lower down under .build/docker/ .These scripts re-used the non-docker scripts inside containers. The docker images are versioned controlled in the same directory as dockerfiles, and tagged by their
md5sums. When running the docker scripts these docker images will, if not in the local registry, will first be attempted to be pulled from dockerhub and only then fallback to be built locally. This allows the dockerfiles to be
modified in existing patches and be run in existing CI as-is. Docker scripts timeout after one hour. virtualenv-clone is used, re-using the python versioned venvs in the image saves time over creating new ones each run. The inner-spliting of docker containers has been removed. It added a lot of complexity, for the most part duplicating the splitting concept, for little performance gain.
Ant target 'generate-unified-test-report' added, used for aggregating test reports and printing a summary.
patch by Mick Semb Wever; reviewed by Brandon Williams, Josh McKenzie, Maxim Muzafarov, Stefan Miklosovic for CASSANDRA-18133
this patch also fixes incorrect visualisation of SSTable min/max local deletion time when no
tombstones are present in a SSTable
patch by Stefan Miklosovic; reviewed by Marcus Eriksson and Brandon Williams for CASSANDRA-18639
Jacek Lewandowski