Commit Graph

290 Commits

Author SHA1 Message Date
Mick Semb Wever f7c11bdcd4
Standalone Jenkinsfile
– ensure test file paths, and their suite names are unique (the latter was broken for cqlshlib and python dtests)
 – removal of properties and system-out in test xml happens in CassandraXMLJUnitResultFormatter
 – new jenkins agent labels and introduce agents sizes
 - ci_summary.html generation script, ref work submitted in apache/cassandra-builds#99
 – fix for stress-test and fqltool-test running on small
 - ant generate-test-report is limited to only running on individual test types (ci_parser.py provides ci_summary.html for the overview now)
 - each cell has a single retry, and the retry will happen on a different agent
 - on ci-cassandra the summary stage happens on the builtin, bc copyArtifacts on 15k+ files takes many hours otherwise
 - test-burn only needs two splits
 - dependency-check is disabled from the lint target until CASSANDRA-19213
 - add $DEBUG env var to in-tree scripts, turns on bash debug
 - fix FBUtilities' handling of gcp cos_containerd (kernel version comes with a trailing '+' character)

 patch by Aleks Volochnev, Mick Semb Wever; reviewed by Aleksandr Volochnev, Josh McKenzie, Maxim Muzafarov, Stefan Miklosovic for CASSANDRA-18594

Co-authored-by: Aleksandr Volochnev <a.volochnev@gmail.com>
Co-authored-by: Mick Semb Wever <mck@apache.org>
Co-authored-by: Josh McKenzie <jmckenzie@apache.org>
Co-authored-by: Artem Chekunov <artem.v.chekunov@gmail.com>
2024-04-04 11:56:36 +02:00
Ariel Weisberg fdfc0019de Merge branch 'cassandra-4.1' into cassandra-5.0 2024-03-22 16:50:22 -04:00
Ariel Weisberg 96692d7206 Merge branch 'cassandra-4.0' into cassandra-4.1 2024-03-22 16:49:15 -04:00
Ariel Weisberg 5dd9213149 Merge branch 'cassandra-3.11' into cassandra-4.0 2024-03-22 16:43:20 -04:00
Ariel Weisberg de2a965a18 Merge branch 'cassandra-3.0' into cassandra-3.11 2024-03-22 16:42:31 -04:00
Ariel Weisberg 38eb339557 Add support for providing nvdDatafeedUrl to OWASP
patch by Ariel Weisberg; reviewed by Berenguer Blasi for CASSANDRA-19484
2024-03-22 16:41:57 -04:00
Branimir Lambov 06ed1afc34 Add an optimized default configuration to tests and make it available for new users
This patch adds a new configuration file, cassandra_latest.yaml, which changes
some of the settings to use new features we usually recommend to users,
including:
- Big Trie-Indexed SSTables
- Trie memtables
- Unified Compaction Strategy
- Storage Attached Index
- Off-heap objects memtables
- Direct-write commit log
- No storage compatibility

These changes are applied in new test targets for unit, in-jvm and python dtests that are
run pre-commit. When changes are made to the new config file, python dtest will use it
directly, but unit and in-jvm dtests require updates, respectively, to latest_diff.yaml
and InstanceConfig.java.

patch by Branimir Lambov; reviewed by Berenguer Blasi for CASSANDRA-18753
2024-03-07 11:08:44 +02:00
Stefan Miklosovic 2482c8a936
Merge branch 'cassandra-4.1' into cassandra-5.0 2024-02-23 14:52:32 +01:00
Andrés de la Peña 8fd44ca8fc Upgrade Python driver to 3.29.0
patch by Andrés de la Peña; reviewed by Berenguer Blasi for CASSANDRA-19245
2024-01-19 17:14:57 +00:00
Mick Semb Wever f54177bb0a
Docker test container fixes: remove unused files/folders in image, and use envs instead of env-file
Adds warning and reduces container requested cpus if over docker daemon limits.

 patch by Mick Semb Wever; reviewed by Brandon Williams for INFRA-25202
2024-01-17 15:23:25 +01:00
Jacek Lewandowski f0ea12c6d7 Add a startup check to fail startup when using invalid configuration with certain Kernel and FS type
Patch by Jacek Lewandowski; reviewed by Maxwell Guo, Stefan Miklosovic for CASSANDRA-19196
2023-12-21 16:42:31 +01:00
Jacek Lewandowski 790798dea0 Merge branch 'cassandra-4.1' into cassandra-5.0
* cassandra-4.1:
  Updated the OWASP plugin to 9.0.5 and refactored dependency checks
2023-12-20 12:59:55 +01:00
Jacek Lewandowski 481d2fd51d Merge branch 'cassandra-4.0' into cassandra-4.1
* cassandra-4.0:
  Updated the OWASP plugin to 9.0.5 and refactored dependency checks
2023-12-20 12:59:54 +01:00
Jacek Lewandowski 891f1119a0 Merge branch 'cassandra-3.11' into cassandra-4.0
* cassandra-3.11:
  Updated the OWASP plugin to 9.0.5 and refactored dependency checks
2023-12-20 12:59:53 +01:00
Jacek Lewandowski 09c46750f6 Merge branch 'cassandra-3.0' into cassandra-3.11
* cassandra-3.0:
  Updated the OWASP plugin to 9.0.5 and refactored dependency checks
2023-12-20 12:59:52 +01:00
Jacek Lewandowski 4c3b96e4cd Updated the OWASP plugin to 9.0.5 and refactored dependency checks
Configured OWASP database to use caching and NVD key. Properties can be
configured in ~/.ant/build.properties and define both the key and
the storage location

Patch by Jacek Lewandowski; reviewed by Maxim Muzafarov, Stefan Miklosovic for CASSANDRA-19146
2023-12-20 12:59:51 +01:00
Brandon Williams a92de92b8c Suppress CVE-2023-6481
Patch by brandonwilliams, reviewed by smiklosovic for CASSANDRA-19184
2023-12-08 08:55:40 -06:00
Brandon Williams fdfc5e614d Merge branch 'cassandra-4.1' into cassandra-5.0 2023-12-06 06:33:36 -06:00
Brandon Williams 13e5956285 Merge branch 'cassandra-4.0' into cassandra-4.1 2023-12-06 06:32:48 -06:00
Brandon Williams 8e5fc74c9a Merge branch 'cassandra-3.11' into cassandra-4.0 2023-12-06 06:32:32 -06:00
Brandon Williams 2e3d7e76f5 Merge branch 'cassandra-3.0' into cassandra-3.11 2023-12-06 06:32:19 -06:00
Brandon Williams a1421ec324 Suppress CVE-2023-6378
Patch by brandonwilliams, reviewed by smiklosovic for CASSANDRA-19142
2023-12-06 06:31:27 -06:00
Mick Semb Wever fe7997884d
Write ccm clusters under test's TMPDIR
This prevents docker overlay file leakage from containers by keeping write-heavy directories in bind mounts (or volumes). It also offers better performance by avoiding the container filesystem.

 patch by Mick Semb Wever; reviewed Brandon Williams for INFRA-25202
2023-12-04 13:26:57 +01:00
Mick Semb Wever 02595b7119
ninja-fix typo "test-trie" 2023-11-15 08:46:15 -08:00
Maxim Muzafarov 072a13e5ef
Exclude net.java.dev.jna:jna dependency from dependencies of org.caffinitas.ohc:ohc-core
patch by Maxim Muzafarov; reviewed by Ariel Weisberg, Ekaterina Dimitrova, Stefan Miklosovic for CASSANDRA-18992
2023-11-15 10:04:56 +01:00
Mick Semb Wever f0655159e6
Add to jenkins pipeline: test-oa, test-system-keyspace-directory, test-trie, jvm-dtest-novnode, jvm-dtest-upgrade-novnode simulator-dtest, dtest-large-novnode, dtest-upgrade-large, dtest-upgrade-novnode, dtest-upgrade-novnode-large.
patch by Mick Semb Wever; reviewed by Berenguer Blasi, Brandon Williams for CASSANDRA-17989, CASSANDRA-18008, CASSANDRA-17145, CASSANDRA-18003
2023-11-14 21:56:57 -05:00
Andrés de la Peña b91c0fcb6e Change cqlsh's min required Python version from 3.6+ to 3.7+
patch by Andrés de la Peña; reviewed by Brandon Williams for CASSANDRA-18960
2023-11-14 11:43:00 +00:00
Jacek Lewandowski 4d61359c21 CASSANDRA-18785: Add support for Sonar analysis
Patch by Jacek Lewandowski; reviewed by Brandon Williams, Maxim Muzafarov, Michael Semb Wever, Stefan Miklosovic for CASSANDRA-18785
2023-11-13 14:48:28 +01:00
Mike Adamson 949b760f55
Add support for a vector search index in SAI
- Adds jbellis/jvector (1.0.2) library for DiskANN based indexes on floating point vectors
- Adds ORDER BY ANN OF capability to do ANN search and order the results by score

 patch by Mike Adamson; reviewed by Andrés de la Peña, Jonathon Ellis for CASSANDRA-18715

Co-authored-by Jonathon Ellis jbellis@gmail.com
Co-authored-by Zhao Yang zhaoyangsingapore@gmail.com
2023-10-29 12:24:00 +01:00
Brandon Williams 3088b8114c Merge branch 'cassandra-4.1' into cassandra-5.0 2023-10-26 06:06:43 -05:00
Brandon Williams d3cc2a65b8 Merge branch 'cassandra-4.0' into cassandra-4.1 2023-10-26 06:05:42 -05:00
Brandon Williams 2fa7c1204c Merge branch 'cassandra-3.11' into cassandra-4.0 2023-10-26 05:59:42 -05:00
Brandon Williams 349ec3e02d Merge branch 'cassandra-3.0' into cassandra-3.11 2023-10-26 05:58:22 -05:00
Brandon Williams 5bf1d2f8d5 Suppress CVE-2023-44487
Patch by brandonwilliams; reviewed by bereng for CASSANDRA-18943
2023-10-26 05:54:19 -05:00
Stefan Miklosovic e2d2bd61f4
ninja-fix nested cassandra clone for dtest jar building re-use check
(git -C … will fall back to parent level .git repo, confusing the check)

patch by Mick Semb Wever; reviewed by Štefan Miklošovič for CASSANDRA-18936,CASSANDRA-18665
2023-10-23 12:28:38 +02:00
mck 0aff36fc74
When generating dtest jars and switching between branches, do git clean before git reset
patch by Mick Semb Wever; reviewed by Stefan Miklosovic for CASSANDRA-18936
2023-10-19 15:26:27 +02:00
Stefan Miklosovic 269285213d
Add versions into Deprecated annotation
patch by Stefan Miklosovic; reviewed by Maxim Muzafarov, Brandon Williams for CASSANDRA-18912

Co-authored-by: Maxim Muzafarov <maxmuzaf@gmail.com>
2023-10-18 19:54:10 +02:00
mck 802bd5fe13
ninja-fix – reusing git clone under build needs reset and permissions
ref: https://the-asf.slack.com/archives/CK23JSY2K/p1697465832946299

 patch by Mick Semb Wever; reviewed by Štefan Miklošovič, Maxim Muzafarov for CASSANDRA-18665
2023-10-17 12:14:21 +02:00
Mick Semb Wever 987d03c142
Fixes for in-tree scripts: offline mode, maybe-build for fqltool-test, jvm-dtest-upgrade
- jvm-dtest-upgrade script running in ci-cassandra.a.o (needs default bash)
 - add offline mode (to avoid `git fetch` before building dtest-jars on other branches),
 - avoid rebuilding in fqltool-test target, as is standard for test targets.
 - python detection and override
 - FileTest cleanup (tmp directory was left unreadable and undeletable)
 - reuse ~/.ccm (like normal)
 - remove support for python 3.6
 - remove docker volumes in image (they are added at run time)
 - remove superfluousjava version debug
 – extra mount for build/tmp as hack to prevent buggy hardlinks on overlay

 patch by Mick Semb Wever; reviewed by Brandon Williams, Ekaterina Dimitrova, Josh McKenzie, Maxim Muzafarov for CASSANDRA-18665
2023-10-16 16:01:28 +02:00
Stefan Miklosovic 5a02e532d4
Merge branch 'cassandra-4.1' into cassandra-5.0 2023-10-10 10:16:22 +02:00
Stefan Miklosovic ea6461b870
Merge branch 'cassandra-4.0' into cassandra-4.1 2023-10-10 10:04:42 +02:00
Maxim Muzafarov 4549b07997
Remove unnecessary reporter-config3 dependency
patch by Maxim Muzafarov; reviewed by Stefan Miklosovic and Brandon Williams for CASSANDRA-18907
2023-10-09 21:40:18 +02:00
Stefan Miklosovic e8c595102a
Exclude unnecessary dependencies introduced by caffeine update to 3.1.8
patch by Stefan Miklosovic; reviewed by Ekaterina Dimitrova, Maxim Muzafarov for CASSANDRA-18906
2023-10-06 11:50:13 +02:00
Maxim Muzafarov f51d9ecbe3
Upgrade Dropwizard Metrics to 4.2.19
patch by Maxim Muzafarov; reviewed by Ekaterina Dimitrova and Stefan Miklosovic for CASSANDRA-14667
2023-10-04 13:41:47 +02:00
Ekaterina Dimitrova eb30005251 Upgrade caffeine cache and fix CIDR permissions cache invalidation
patch by Ekaterina Dimitrova; reviewed by Jacek Lewandowski, Ben Manes, Yifan Cai, Shailaja Koppu for CASSANDRA-18805
2023-10-03 16:14:34 -04:00
Ekaterina Dimitrova b07034eeb7 Merge branch 'cassandra-4.1' into cassandra-5.0 2023-10-03 11:12:14 -04:00
Stefan Miklosovic 449343138d
Merge branch 'cassandra-4.1' into cassandra-5.0 2023-09-27 21:01:44 +02:00
Maxim Muzafarov a580b79d16 Upgrade slf4j-api to 1.7.36
Patch by mmuzaf; reviewed by brandonwilliams and edimitrova for
CASSANDRA-18882
2023-09-27 13:30:08 -05:00
Mick Semb Wever f65eeae92b
Add python upgrade dtests with vnodes
patch by Mick Semb Wever; reviewed by Berenguer Blasi for CASSANDRA-18842
2023-09-27 11:59:37 +02:00
Jacek Lewandowski 54ef3a1c7e Merge branch 'cassandra-4.1' into cassandra-5.0
* cassandra-4.1:
  JMH improvements - faster build and async profiler
2023-09-25 12:39:39 +02:00