patch by Vineet Gali; reviewed by Brandon Williams, Brad Schoening for CASSANDRA-17667
Co-authored-by: Attila Homoki <biotep@yahoo.com>
Co-authored-by: Brad Schoening <bschoeni@gmail.com>
In addition - Fixed Three Valued Logic
patch by Piotr Kolaczkowski; reviewed by Benjamin Lerer, Ekaterina Dimitrova, Andres de la Pena for CASSANDRA-18584
Co-authored-by: Benjamin Lerer <b.lerer@gmail.com>
Co-authored by: Ekaterina Dimitrova<ekaterina.dimitrova@datastax.com>
– ensure test file paths, and their suite names are unique (the latter was broken for cqlshlib and python dtests)
– removal of properties and system-out in test xml happens in CassandraXMLJUnitResultFormatter
– new jenkins agent labels and introduce agents sizes
- ci_summary.html generation script, ref work submitted in apache/cassandra-builds#99
– fix for stress-test and fqltool-test running on small
- ant generate-test-report is limited to only running on individual test types (ci_parser.py provides ci_summary.html for the overview now)
- each cell has a single retry, and the retry will happen on a different agent
- on ci-cassandra the summary stage happens on the builtin, bc copyArtifacts on 15k+ files takes many hours otherwise
- test-burn only needs two splits
- dependency-check is disabled from the lint target until CASSANDRA-19213
- add $DEBUG env var to in-tree scripts, turns on bash debug
- fix FBUtilities' handling of gcp cos_containerd (kernel version comes with a trailing '+' character)
patch by Aleks Volochnev, Mick Semb Wever; reviewed by Aleksandr Volochnev, Josh McKenzie, Maxim Muzafarov, Stefan Miklosovic for CASSANDRA-18594
Co-authored-by: Aleksandr Volochnev <a.volochnev@gmail.com>
Co-authored-by: Mick Semb Wever <mck@apache.org>
Co-authored-by: Josh McKenzie <jmckenzie@apache.org>
Co-authored-by: Artem Chekunov <artem.v.chekunov@gmail.com>
patch by Lorina Poland; reviewed by Michael Semb Wever for CASSANDRA-19249, CASSANDRA-18990, CASSANDRA-15719
Co-authored-by: Annette Dennis <annette.dennis@datastax.com>
Co-authored-by: Mohsin Medmood <mohsin85mehmood@gmail.com>
Co-authored-by: Josh Wong <joshuarwong@outlook.com>
This patch adds a converter that allows parsing vector literals
passed via csv files to the COPY command.
patch by Szymon Miezal; reviewed by Andrés de la Peña, Stefan Miklosovic and Maxwell Guo for CASSANDRA-19118
This prevents docker overlay file leakage from containers by keeping write-heavy directories in bind mounts (or volumes). It also offers better performance by avoiding the container filesystem.
patch by Mick Semb Wever; reviewed Brandon Williams for INFRA-25202
An overview of the core components can be found in the included
TransactionalClusterMetadata.md
patch by Alex Petrov, Marcus Eriksson and Sam Tunnicliffe; reviewed by
Alex Petrov, Marcus Eriksson and Sam Tunnicliffe for CASSANDRA-18330
Co-authored-by: Marcus Eriksson <marcuse@apache.org>
Co-authored-by: Alex Petrov <oleksandr.petrov@gmail.com>
Co-authored-by: Sam Tunnicliffe <samt@apache.org>
patch by Andrés de la Peña and Jeremiah Jordan; reviewed by Brandon Williams for CASSANDRA-18960
Co-authored-by: Jeremiah Jordan <jeremiah@datastax.com>
Co-authored-by: Andrés de la Peña <a.penya.garcia@gmail.com>
This patch removes deprecated chunk_length_kb and sstable_compression in favor of
chunk_length_in_kb and class parameters.
The only deprecated compression parameter left, crc_check_chance (which is configurable outside
of compression parameters as a standalone table parameter) is meant to be treated separately.
patch by Stefan Miklosovic; reviewed by Maxim Muzafarov and Brandon Williams for CASSANDRA-18742
See .build/README.md .Build and test scripts cover all CI activities in an CI agnostic manner. Scripts are found in non-docker and dockerised variants.
The non-docker scripts are found first under .build/ .These interact directly with ant, providing additional checks and environment setup. It is intended that what can be pushed down into the base ant build layer should be. The
debian and redhat build scripts are hidden under .build/docker because they are generally not usuable outside of their corresponding linux dists, and we don't want to encourage anything to the contrary. Bring the artifact/deb/rpm
build scripts (and associated docker images) from cassandra-builds repo to the .build directory. These packaging script for debian and redhat are now separated from the artifacts scripts. An additional build script check-code.sh
has been added that is solely responsible for all linter and code checks (includes owasp dependency checker). Only JDK11 and JDK17 are supported (as JDK8 is soon to be dropped). Scripts do not clean (or realclean) so that
scripts can be better pipelined, but `ant jar` was required to be added to circleci and ci-cassandra.a.o cqlsh-tests. Scripts can define a custom BUILD_DIR (build.dir) which can permit running parrallel builds off the same source
(though not all tests support this, or their support for it has atrophied). All build and test artifacts, results, and logs are found under build/ (or the BUILD_DIR). The build local venv and .ccm directories are now also
created under build (or BUILD_DIR). The test venv, logs and results are wiped each run. Use a separate mktemp directory each run, under /tmp (respect /tmp responsibility, and python fails when tmpdir is on a docker volume).
Don't set CASSANDRA_HOME when running python dtests (and fail-fast if `ant artifacts` was used to build instead of `ant jar`).
The docker scripts are found lower down under .build/docker/ .These scripts re-used the non-docker scripts inside containers. The docker images are versioned controlled in the same directory as dockerfiles, and tagged by their
md5sums. When running the docker scripts these docker images will, if not in the local registry, will first be attempted to be pulled from dockerhub and only then fallback to be built locally. This allows the dockerfiles to be
modified in existing patches and be run in existing CI as-is. Docker scripts timeout after one hour. virtualenv-clone is used, re-using the python versioned venvs in the image saves time over creating new ones each run. The inner-spliting of docker containers has been removed. It added a lot of complexity, for the most part duplicating the splitting concept, for little performance gain.
Ant target 'generate-unified-test-report' added, used for aggregating test reports and printing a summary.
patch by Mick Semb Wever; reviewed by Brandon Williams, Josh McKenzie, Maxim Muzafarov, Stefan Miklosovic for CASSANDRA-18133
Allows to attach the native masking functions and UDFs to the definitions of
table columns in the schema, as defined by CEP-20.
The functions masking a column can be specified on CREATE TABLE queries, right
at the end of the column definition. The mask of a column can also be changed
or dropped with an ALTER TABLE query. Once a column is masked, SELECT queries
will always return the masked value of the column. That masking is done on the
coordinator, at the end of the query execution. Thus, masking won't affect any
filters or ordering, which would be based on the clear values of the masked
columns. Column masks are stored on the table system_schema.column_masks.
A new UNMASK permission allows to see the clear data of columns with an
attached mask. Also, a new SELECT_MASKED permission allows to run SELECT
queries restricting the clear values of masked columns. Superusers have both
permissions by default, whereas regular users don't have them.
Dynamic data masking can be enabled/disabled with the config property
dynamic_data_masking_enabled in cassandra.yaml. It is disabled by default.
This is the combination of multiple tickets:
* Add masking functions to column metadata (CASSANDRA-18068)
* Add UNMASK permission (CASSANDRA-18069)
* Add SELECT_MASKED permission (CASSANDRA-18070)
* Add support for using UDFs as masking functions (CASSANDRA-18071)
* Add feature flag for dynamic data masking (CASSANDRA-18316)
patch by Andrés de la Peña; reviewed by Benjamin Lerer and Berenguer Blasi for CASSANDRA-17940
Removes the use of CASSANDRA_USE_JDK11 and introduces the build.xml properties `java.default` and `java.supported`.
patch by Mick Semb Wever; reviewed by Ekaterina Dimitrova for CASSANDRA-18179
patch by Brad Schoening; reviewed by Stefan Miklosovic and Brandon Williams for CASSANDRA-17832
Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
Fix CQLSHLIB tests config and Python DTests split; CQLSHLIB tests should run with NUM_TOKENS=16
patch by Ekaterina Dimitrova; reviewed by Andres de la Pena for CASSANDRA-18001
vgali7