As part of a broader effort to decouple java driver code from the
server code, this moves sstableloader to its own tools directory.
As sstableloader is also used as a library (CASSANDRA-10637), added
a new artifact 'cassandra-sstableloader' that will get deployed to
maven along with 'cassandra-all'.
While I expect this is likely a niche use case, this will allow users
to continue using BulkExport as a library.
Moves sstableloader-specific targets to its own build.xml in
tools/sstableloader/build.xml.
Also updates IDE project files and circleci to utilize new
sstableloader-specific targets.
patch by Andy Tolbert; reviewed by Stefan Miklosovic and Mick Semb Wever for CASSANDRA-20328
Also
– increase splits for jvm-dtests, container size and docker limits for simulator-dtest
– print exit status in .build/docker/run-tests.sh and .build/docker/_docker_init_tests.sh
– add a summary list of test failures in ci_summary.html
– in ubuntu2004_test.docker make sure /home/cassandra exists and has correct perms (from Marcuse)
– when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile generateTestReports() with manual wget of test files (copyArtifacts is notoriously slow, >4hrs in this case)
– copy ci_summary.html and results_details.tar.xz to nightlies
– capture generateTestReports logs
– stream xz where possible, instead of xz compressing afterwards
– fix ant version in centos7-build.docker
– remove docker login (was meangingless, if credentials exist then docker is logged in)
– prefetch docker images from jfrog (to reduce dockerhub pull rate limits), relates to CASSANDRA-18931
– use scripts from cassandra-builds to clean and report on agents, relates to CASSANDRA-18130
– use groovy elvis operators to avoid NPEs/failures on blank param values
– parallel jfrog pulls and cache it in jfrog too
– pin alpine docker image version
– don't stash .git (was causing AccessDeniedException on some unstash), needed fix so deb/rpm packaging worked on not-git work directories
– durabilityHint needs PERFORMANCE_OPTIMIZED, controller becomes scale bottleneck and breaks without it
patch by Mick Semb Wever; reviewed by Brandon Williams for CASSANDRA-19558
– ensure test file paths, and their suite names are unique (the latter was broken for cqlshlib and python dtests)
– removal of properties and system-out in test xml happens in CassandraXMLJUnitResultFormatter
– new jenkins agent labels and introduce agents sizes
- ci_summary.html generation script, ref work submitted in apache/cassandra-builds#99
– fix for stress-test and fqltool-test running on small
- ant generate-test-report is limited to only running on individual test types (ci_parser.py provides ci_summary.html for the overview now)
- each cell has a single retry, and the retry will happen on a different agent
- on ci-cassandra the summary stage happens on the builtin, bc copyArtifacts on 15k+ files takes many hours otherwise
- test-burn only needs two splits
- dependency-check is disabled from the lint target until CASSANDRA-19213
- add $DEBUG env var to in-tree scripts, turns on bash debug
- fix FBUtilities' handling of gcp cos_containerd (kernel version comes with a trailing '+' character)
patch by Aleks Volochnev, Mick Semb Wever; reviewed by Aleksandr Volochnev, Josh McKenzie, Maxim Muzafarov, Stefan Miklosovic for CASSANDRA-18594
Co-authored-by: Aleksandr Volochnev <a.volochnev@gmail.com>
Co-authored-by: Mick Semb Wever <mck@apache.org>
Co-authored-by: Josh McKenzie <jmckenzie@apache.org>
Co-authored-by: Artem Chekunov <artem.v.chekunov@gmail.com>
This patch adds a new configuration file, cassandra_latest.yaml, which changes
some of the settings to use new features we usually recommend to users,
including:
- Big Trie-Indexed SSTables
- Trie memtables
- Unified Compaction Strategy
- Storage Attached Index
- Off-heap objects memtables
- Direct-write commit log
- No storage compatibility
These changes are applied in new test targets for unit, in-jvm and python dtests that are
run pre-commit. When changes are made to the new config file, python dtest will use it
directly, but unit and in-jvm dtests require updates, respectively, to latest_diff.yaml
and InstanceConfig.java.
patch by Branimir Lambov; reviewed by Berenguer Blasi for CASSANDRA-18753
- Rename and refactor a bit property for overriding storage compatibility mode
- Fix streaming for tools
- consider bulkloader as a client instead of tool
Patch by Jacek Lewandowski; reviewed by Berenguer Blasi, Branimir Lambov for CASSANDRA-19126
Configured OWASP database to use caching and NVD key. Properties can be
configured in ~/.ant/build.properties and define both the key and
the storage location
Patch by Jacek Lewandowski; reviewed by Maxim Muzafarov, Stefan Miklosovic for CASSANDRA-19146
An overview of the core components can be found in the included
TransactionalClusterMetadata.md
patch by Alex Petrov, Marcus Eriksson and Sam Tunnicliffe; reviewed by
Alex Petrov, Marcus Eriksson and Sam Tunnicliffe for CASSANDRA-18330
Co-authored-by: Marcus Eriksson <marcuse@apache.org>
Co-authored-by: Alex Petrov <oleksandr.petrov@gmail.com>
Co-authored-by: Sam Tunnicliffe <samt@apache.org>
- jvm-dtest-upgrade script running in ci-cassandra.a.o (needs default bash)
- add offline mode (to avoid `git fetch` before building dtest-jars on other branches),
- avoid rebuilding in fqltool-test target, as is standard for test targets.
- python detection and override
- FileTest cleanup (tmp directory was left unreadable and undeletable)
- reuse ~/.ccm (like normal)
- remove support for python 3.6
- remove docker volumes in image (they are added at run time)
- remove superfluousjava version debug
– extra mount for build/tmp as hack to prevent buggy hardlinks on overlay
patch by Mick Semb Wever; reviewed by Brandon Williams, Ekaterina Dimitrova, Josh McKenzie, Maxim Muzafarov for CASSANDRA-18665
This patch also removes compress-lzf library (leftover from CASSANDRA-12229)
as well as it makes byte-buddy dependencies to be test scoped.
patch by Stefan Miklosovic; reviewed by Michael Semb Wever for CASSANDRA-18877
Brandon Williams