Commit Graph

29072 Commits

Author SHA1 Message Date
vgali7 b4b9f31983
Migrate Python optparse to argparse
patch by Vineet Gali; reviewed by Stefan Miklosovic and Brandon Williams for CASSANDRA-17914
2023-07-25 09:54:09 +02:00
Branimir Lambov a565711056 Fix data corruption handling issues
Treat AssertionError as corruption, assert positive deletion timestamps and
TTLs and treat localDeletionTime < TTL as invalid.

patch by Branimir Lambov; reviewed by Berenguer Blasi for CASSANDRA-18676
2023-07-25 08:07:03 +03:00
Bereng 2a458ba01d Improved DeletionTime serialization 2023-07-26 10:47:22 +02:00
Ekaterina Dimitrova d08fd1ebd8 Revert "Fix MemtableSizeTestBase#testSize (now MemtableSizeTest#testSize) was not running in CI"
Not needed
patch by Ekaterina Dimitrova; reviwed by Brandon Williams for CASSANDRA-18329

This reverts commit 6b885a44c3.
2023-07-25 17:45:51 -04:00
Maulin Vasavada bbee4c87c4 Add equals/hashCode override for ServerEncryptionOptions
Patch by Maulin Vasavada; reviewed by brandonwilliams and smiklosovic
for CASSANDRA-18428
2023-07-25 16:13:56 -05:00
Ekaterina Dimitrova c1bf002234 Fix building dtest jar
patch by Jacek Lewandowski; reviewed by Ekaterina Dimitrova and Brandon Williams for CASSANDRA-18690
2023-07-25 16:14:34 -04:00
Ekaterina Dimitrova b92184e90c Update SyncUtil for JDK11 and JDK17
Remove --add-opens java.base/java.nio=ALL-UNNAMED which is not needed anymore

patch by Ekaterina Dimitrova; reviewed by Jacek Lewandowki for CASSANDRA-17909
2023-07-25 15:33:39 -04:00
Ekaterina Dimitrova 5722191a38 Fix generate.sh and remove dead J8 config files
patch by Ekaterina Dimitrova; reviewed by Brandon Williams for CASSANDRA-18255
2023-07-25 12:20:08 -04:00
Stefan Miklosovic ac26b44f9d
Merge branch 'cassandra-4.1' into trunk 2023-07-25 10:43:18 +02:00
Ekaterina Dimitrova b61bd93e57 Upgrade ecj version to 3.33.0
Bump source and target version to 11 for Java UDFs
ICompilationUnit has new default methods

patch by Ekaterina Dimitrova; reviewed by Andres de la Pena and Berenguer Blasi for CASSANRA-18190
2023-07-24 21:49:43 -04:00
Daniel Jatnieks fe8a6eb70f CASSANDRA-18180 Fix bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
Fix ClassCastException  from jdk GaloisCounterMode when using JDK17 provider

Add javac exports to build.xml for java.base/jdk.internal.ref and java.base/sun.nio.ch

Replace checktestnameshelper ant task in build.xml with _check-test-names java target to be able to pass jvm args to TestNameCheckTask

patch by Dan Janieks; revewed by Ekaterina Dimitrova and Andres de la Pena for CASSANDRA-18180
2023-07-24 20:58:49 -04:00
Ekaterina Dimitrova 02f6353e1f Ninja fix: add fake eclipse-warnings task so that Jenkins doesn't fail until the build scripts are updated
This fix was already tested with Jenkins in 18190 which will be committed later
2023-07-24 16:10:22 -04:00
Ekaterina Dimitrova 3755934e52 Drop JDK8 and add JDK17, remove eclipse-warnings in favor of Checker Framework and upgrade checkstyle
patch by Ekaterina Dimitrova; reviewed by Jeremiah Jordan, Berenguer Blasi, Michael Semb Wever and Jacek Lewandowski for CASSANDRA-18255
2023-07-24 15:16:53 -04:00
Stefan Miklosovic 08a9dd805e
Merge branch 'cassandra-4.1' into trunk 2023-07-24 13:34:15 +02:00
Stefan Miklosovic 9bc0f89308
increment version to 4.1.4 2023-07-24 13:33:24 +02:00
KowalczykBartek 8faa2a2c69 Remove WaitingOnFreeMemtableSpace and DroppedMutations metrics
Patch by Bartlomiej, reviewed by bereng and brandonwilliams for
CASSANDRA-18298
2023-07-24 05:53:39 -05:00
Benjamin Lerer 600f4d9a69 Upgrade Jamm version to 0.4.0
This upgrade also fixes issues with PhantomReferences and the test problems from CASSANDRA-17884 anad CASSANDRA-16304

patch by Benjamin Lerer; reviewed by Ekaterina Dimitrova for CASSANDRA-18239
2023-07-22 14:55:49 -04:00
Ekaterina Dimitrova 6b885a44c3 Fix MemtableSizeTestBase#testSize (now MemtableSizeTest#testSize) was not running in CI
patch by Ekaterina Dimitrova; reviewed by Brandon Williams and Michael Semb Wever for CASSANDRA-18329
2023-07-22 14:41:43 -04:00
Benjamin Lerer b41e2f5b80 Fix eclipse problems
patch by Bejamin Lerer; reviewed by Ekaterina Dimitrova for CASSANDRA-18329
2023-07-22 14:29:22 -04:00
Mick Semb Wever 3bf9bd6831
Remove legacy 3.0/3.11 buffer pool metrics
patch by Mick Semb Wever; reviewed by Brandon Williams, Maxim Muzafarov for CASSANDRA-18313
2023-07-22 11:35:44 +02:00
Jacek Lewandowski ad26ffcd57 Run checks in a separate task and fix build warnings
Patch by Jacek Lewandowski; reviewed by Mick Semb Wever and Stefan Miklosovic for CASSANDRA-18618
2023-07-21 05:57:37 +02:00
Stefan Miklosovic c273017b25
Add AzureSnitch
As we were implementing the snitch itself, we noticed that the constructors
for cloud-based snitches are unnecessarilly complicated and
we took the opportunity to make them simpler.

patch by Stefan Miklosovic; reviewed by German Eichberger and Jacek Lewandowski for CASSANDRA-18646
2023-07-20 21:35:07 +02:00
Mick Semb Wever 6643ea7551
Add the dtest-upgrade-large (and other) test type for resource-intensive Python upgrade tests
Fixes:
 - regular expressions when filtering on tests to run, and exit when no match
 - safety when removing (cleaning up) directories (${DIST_DIR} and ${TMPDIR})
 - pre-condition `ant jar` has been used and not `ant artifacts`
 - compatibility with git worktrees (the original git working directory needs to be mounted in its existing path in docker)
 - fix tarball for ccm logs
 - add `--keep-failed-test-dir` to dtest-upgrade test types
 - only do `docker login` if `docker pull` is going to be called
 - unset CASSANDRA_HOME when running python dtests

 patch by Mick Semb Wever; reviewed by Brandon Williams, Jacek Lewandowski for CASSANDRA-18567, CASSANDRA-18499
2023-07-20 18:40:40 +02:00
Branimir Lambov 957eca2fb5 Implementation of the Unified Compaction Strategy as described in CEP-26
The approach is documented in the included UnifiedCompactionStrategy.md.

Also included:
- Generalize prettyPrint to prettyPrintBinary (with 2^10 multiplier,
  e.g. MiB) and prettyPrintDecimal (with 1000 multiplier, e.g. ms)
  that cover the full range of double, and add a parseHumanReadable
  method that can read both.

- CASSANDRA-18123: Fix invalid reuse of metadata collector during flushing

- Fix invalid token range splitting with non-unit weights

- Add flushSizeOnDisk metric

- Add ability to change compaction default in YAML

patch by Alex Sorokoumov, Branimir Lambov, Dimitar Dimitrov and Stefania Alborghetti
reviewed by Alex Sorokoumov, Jaroslaw Grabowski and Maxwell Guo for CASSANDRA-18397
2023-07-19 11:22:32 +03:00
Branimir Lambov e9198d6a66 Fix incorrect value of preload flag for opening BTI partition indexes
patch by Branimir Lambov; reviewed by Jacek Lewandowski for CASSANDRA-18677
2023-07-19 05:51:41 +03:00
Stefan Miklosovic 707da8c69f
Merge branch 'cassandra-4.1' into trunk 2023-07-19 15:55:04 +02:00
Stefan Miklosovic 2a4cd36475
Prepare debian changelog for 4.1.3 2023-07-18 22:04:57 +02:00
timothytu12 00e5431d64 Upgrade commons cli to 1.5.0
Patch by Timothy Tu; reviewed by bereng and brandonwilliams for
CASSANDRA-18659
2023-07-18 06:07:29 -05:00
Stefan Miklosovic b2f08da641
Merge branch 'cassandra-4.1' into trunk 2023-07-18 11:31:56 +02:00
Stefan Miklosovic 292e26842a
Merge branch 'cassandra-4.0' into cassandra-4.1 2023-07-18 11:29:43 +02:00
Stefan Miklosovic 2584f4f070
increment version to 4.0.12 2023-07-18 11:26:19 +02:00
Brandon Williams d6b305421a Merge branch 'cassandra-4.1' into trunk 2023-07-17 11:37:16 -05:00
Brandon Williams 7524ed5bfe Merge branch 'cassandra-4.0' into cassandra-4.1 2023-07-17 11:32:49 -05:00
Marianne Lyne Manaog a7bf85c4f3 Add jvm/test args to microbench
Patch by Marianne Lyne Manaog; reviewed by brandonwilliams and
edimitrova for CASSANDRA-18658
2023-07-17 11:29:19 -05:00
Daniel Jatnieks aac070681b Disable the deprecated keyspace/table thresholds and convert them to guardrails
patch by Daniel Jatnieks; reviewed by Andrés de la Peña and Brandon Williams for CASSANDRA-18617
2023-07-14 12:23:04 +01:00
Stefan Miklosovic 903857b4ef
Deprecate CloudstackSnitch and remove duplicate code in snitches
The patch also refactors existing cloud snitches to get rid of the duplicate code,
this is the logical follow-up of CASSANDRA-16555 where AbstractCloudMetadataServiceConnector was introduced.

patch by Stefan Miklosovic; reviewed by Jacek Lewandowski, Jackson Fleming and Maxwell Guo for CASSANDRA-18438
2023-07-14 12:38:30 +02:00
Bereng 6f486f5847 Merge branch 'cassandra-4.1' into trunk
* cassandra-4.1:
  Test failure: org.apache.cassandra.distributed.test.ByteBuddyExamplesTest.countTest
2023-07-14 08:24:28 +02:00
Bereng 2af60dced2 Test failure: org.apache.cassandra.distributed.test.ByteBuddyExamplesTest.countTest 2023-07-14 08:21:11 +02:00
Jacek Lewandowski 0910901d7b Merge branch 'cassandra-4.1' into trunk
* cassandra-4.1:
  Moved jflex from runtime to build dependencies
2023-07-13 15:33:13 +02:00
Jacek Lewandowski 17afd5e6be Merge branch 'cassandra-4.0' into cassandra-4.1
* cassandra-4.0:
  Moved jflex from runtime to build dependencies
2023-07-13 15:33:12 +02:00
Jacek Lewandowski 4f0865f769 Merge branch 'cassandra-3.11' into cassandra-4.0
* cassandra-3.11:
  Moved jflex from runtime to build dependencies
2023-07-13 15:33:11 +02:00
Jacek Lewandowski 32afc93b0d Moved jflex from runtime to build dependencies
Patch by Jacek Lewandowski; reviewed by Berenguer Blasi, Mick Semb Wever and Stefan Miklosovic for CASSANDRA-18664
2023-07-13 15:33:10 +02:00
Mick Semb Wever f038059e89
ninja-fix test*some targets need to pass a relative path to outfile
patch by Mick Semb Wever; reviewed by Stefan Miklosovic, Berenguer Blasi, Jacek Lewandowski for CASSANDRA-18133
2023-07-13 13:55:09 +02:00
Stefan Miklosovic c10592eddd
Merge branch 'cassandra-4.1' into trunk 2023-07-12 22:02:16 +02:00
Stefan Miklosovic e4dd486734
Merge branch 'cassandra-4.0' into cassandra-4.1 2023-07-12 22:01:53 +02:00
Stefan Miklosovic f8584b943e
Prepare debian changelog for 4.0.11 2023-07-12 21:47:08 +02:00
Mick Semb Wever f5df4b219e
Move build and test scripts in-tree, under .build/
See .build/README.md  .Build and test scripts cover all CI activities in an CI agnostic manner. Scripts are found in non-docker and dockerised variants.

The non-docker scripts are found first under .build/ .These interact directly with ant, providing additional checks and environment setup. It is intended that what can be pushed down into the base ant build layer should be. The
debian and redhat build scripts are hidden under .build/docker because they are generally not usuable outside of their corresponding linux dists, and we don't want to encourage anything to the contrary.  Bring the artifact/deb/rpm
build scripts (and associated docker images) from cassandra-builds repo to the .build directory.  These packaging script for debian and redhat are now separated from the artifacts scripts. An additional build script check-code.sh
has been added that is solely responsible for all linter and code checks (includes owasp dependency checker).  Only JDK11 and JDK17 are supported (as JDK8 is soon to be dropped).  Scripts do not clean (or realclean) so that
scripts can be better pipelined, but `ant jar` was required to be added to circleci and ci-cassandra.a.o cqlsh-tests.  Scripts can define a custom BUILD_DIR (build.dir) which can permit running parrallel builds off the same source
(though not all tests support this, or their support for it has atrophied).  All build and test artifacts, results, and logs are found under build/ (or the BUILD_DIR).  The build local venv and .ccm directories are now also
created under build (or BUILD_DIR).  The test venv, logs and results are wiped each run.  Use a separate mktemp directory each run, under /tmp (respect /tmp responsibility, and python fails when tmpdir is on a docker volume).
Don't set CASSANDRA_HOME when running python dtests (and fail-fast if `ant artifacts` was used to build instead of `ant jar`).

The docker scripts are found lower down under .build/docker/ .These scripts re-used the non-docker scripts inside containers. The docker images are versioned controlled in the same directory as dockerfiles, and tagged by their
md5sums.  When running the docker scripts these docker images will, if not in the local registry, will first be attempted to be pulled from dockerhub and only then fallback to be built locally.  This allows the dockerfiles to be
modified in existing patches and be run in existing CI as-is.  Docker scripts timeout after one hour.  virtualenv-clone is used, re-using the python versioned venvs in the image saves time over creating new ones each run.  The inner-spliting of docker containers has been removed.  It added a lot of complexity, for the most part duplicating the splitting concept, for little performance gain.

Ant target 'generate-unified-test-report' added, used for aggregating test reports and printing a summary.

 patch by Mick Semb Wever; reviewed by Brandon Williams, Josh McKenzie, Maxim Muzafarov, Stefan Miklosovic for CASSANDRA-18133
2023-07-12 12:59:37 +02:00
Andrés de la Peña ddbc52990f Add support for vectors in UDFs
patch by Andrés de la Peña; reviewed by Berenguer Blasi and Maxwell Guo for CASSANDRA-18613
2023-07-12 11:43:58 +01:00
Stefan Miklosovic bf11c9ae03
Merge branch 'cassandra-4.1' into trunk 2023-07-12 12:22:54 +02:00
Stefan Miklosovic d320cef8f9
Merge branch 'cassandra-4.0' into cassandra-4.1 2023-07-12 12:21:03 +02:00