By default the expiry time is calculated on each peer independently. It can be
made to converge by disabling gossip quarantine using the configuration setting
gossip_quarantine_disabled or via a hotprop on GossiperMBean.
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-21035
* cassandra-5.0:
Fix CQLSSTableWriter serialization of vector of date and time patch by Lukasz Antoniak; reviewed by Andres de la Pena, Yifan Cai for CASSANDRA-20979
Use a plain loop to check if it is ASCII symbol before going into more complicated UTF8 parsing.
Avoid ValueAccessor to get extra boost for the ASCII check, especially in non-monomorphic cases.
Patch by Dmitry Konstantinov; reviewed by Jyothsna Konisa, Stefan Miklosovic for CASSANDRA-21075
Additionally replace List with array for bind values (we know the size in advance during a decoding), so in total: List<List> is replaced with byte[][] QueryOptions classes support both ways to get values now: using an old API with ByteBuffer and a new API with byte[].
Patch by Dmitry Konstantinov; reviewed by Michael Semb Wever for CASSANDRA-20166
Converting collections or UDTs to raw bytes is nonsensical - it
allows reading raw serialized bytes which have no meaningful
interpretation.
patch by Mikołaj Diakowski; reviewed by Stefan Miklosovic, Brandon Williams for CASSANDRA-20982
Move TableMetadataRef.get to init part of bulk operations (such as flush, compaction, scrab)
Cache TableMetadata value within TableMetadataRef to reduce overheads for write operations, same for KeyspaceMetadata
Patch by Dmitry Konstantinov; reviewed by Marcus Eriksson for CASSANDRA-20465
* Better handling of DOWN unupgraded nodes
* Test that aborted CMS initialization cleans up state properly
* Clean up orphaned PreInitialize entries in the log on bounce
* Unconditionally reset initiator during abort
* Ensure that metadata log entries aren't exchanged before CMS
initialization is complete
* Precalculate common serialization version, excluding non-upgraded and
LEFT nodes
* Decide if metadata-impacting upgrade is in progress using min common
version
* Add metadata identifier to nodetool cms output
Patch by Sam Tunnicliffe and Marcus Eriksson; reviewed by Sam
Tunnicliffe and Marcus Eriksson for CASSANDRA-21036
Co-authored-by: Marcus Eriksson <marcuse@apache.org>
Co-authored-by: Sam Tunnicliffe <samt@apache.org>
For GC like ShenandoahGC/ZGC etc., there are GC events that do not have STW pauses (Concurrent phases).
Operator might find it reasonable to use lower thresholds for events require STW pauses and higher thresholds for concurrent phases.
gc_concurrent_phase_log_threshold and gc_concurrent_phase_warn_threshold configuration options are introduced in cassandra.yaml
Patch by Yuqi Yan; reviewed by Dmitry Konstantinov, Stefan Miklosovic for CASSANDRA-20980
Co-authored-by: Stefan Miklosovic
- Estimate memory required to allocate for applying a partition update into a memtable and allocate this memory in one shot, then use it as a request-local SLAB
- Reduce contention by switching MemtableAllocator.SubAllocator#owns from updates via AtomicLongFieldUpdater to LongAdder usage. MemtableAllocator.SubAllocator#acquired(..) method updates "owns" value but does not use the updated result.
- Reduce contention by replacing of CAS loop in MemtablePool.SubPool#tryAllocate with allocatedUpdater.addAndGet(this, size)
Patch by Dmitry Konstantinov; reviewed by Michael Semb Wever for CASSANDRA-20226
Before the fix during a capacity extension BufferPoolAllocator returned to BufferPool a sliced ByteBuffer wrapper object instead of the originally allocated one, so the ByteBuffer was not recycled by BufferPool
Adjust BufferPoolAllocatorTest to test the ByteBuf capacity extension with a real BufferPool behavior
Patch by Dmitry Konstantinov; reviewed by Michael Semb Wever for CASSANDRA-20753
* cassandra-5.0:
ninja-fix – Fix eclipse-warnings error for CASSANDRA-19564
ReadCommandController should close fast to avoid deadlock when building secondary index
* cassandra-4.1:
ninja-fix – Fix eclipse-warnings error for CASSANDRA-19564
ReadCommandController should close fast to avoid deadlock when building secondary index
Codahale metrics do not provide the ability to create custom metric implementations, so we have to inherit from Codahale classes.
For better cache locality rate and counter values are extracted to a common thread-local arrays.
Threads death is tracked using 2 approaches: FastThreadLocal.onRemoval callback and phantom references to Thread objects.
Phantom references are used to track aliveness of metric users and reusing of metric IDs.
Patch by Dmitry Konstantinov; reviewed by Benedict Elliott Smith for CASSANDRA-20250
This is backport of CASSANDRA-18875 to 5.0 where we upgrade 2.1 of snakeyaml and bump jackson-dataformat-yaml to 2.19.2.
The change in trunk (5.1) is about excluding Jackson dependencies from wiremock so 2.19.2 of Jackson is used everywhere in test as well,
otherwise there would be Jackson annotations of 2.13.4 used.
patch by Raymond Huffman; reviewed by Mick Semb Wever and Stefan Miklosovic for CASSANDRA-18875
Co-authored-by: Griffin Davis <gcd@ibm.com>
The patch for 5.0 is preserving old behavior, it is possible to turn on JSON representation by a system property.
The patch for trunk is by default transforming collections to JSON string but there is the property (same as in 5.0) which has default to be set to true.
patch by Marko Tsymbaluk; reviewed by Paulo Motta, Stefan Miklosovic for CASSANDRA-20827
Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
The length of table names was not controlled. This is likely due to
confusion between validation methods with similar names. As result
creating tables with too long names led to the too long file name
exceptions during table creations.
This commit adds a validation of table name lengths to avoid the too
long file name errors. The validation length is based on how the table
name is used to create file/directory names, and needs to be exact to
prevent the too long file name exception, but allow all other table
names, which didn't lead to the too long file name exception. This
length limit is different from the existing name length limit of 48
characters used by common validation functions.
Thus, this commit moves out the length validation from the validation
methods into a separate length validation method, so the errors on
names are more specific. The non-length validation methods combined
into a single method, which checks for empty names and valid characters.
New constants are added for the length limits.
Table name related code are moved into methods in TableMetadata class,
so their semantics are more clear and to allow reuse, e.g., in
asserting the table name length constant.
Tests are added for the long table names and non-alphanumeric names.
Keyspace name validation function is now shared between two classes and
a unit test of it is added.
Patch by Ruslan Fomkin; reviewed by Piotr Kołaczkowski, Dmitry
Konstantinov, Maxwell Guo for CASSANDRA-20389
patch by Caleb Rackliffe; reviewed by Marcus Eriksson for CASSANDRA-20887
Co-authored-by: Caleb Rackliffe <calebrackliffe@gmail.com>
Co-authored-by: Marcus Eriksson <marcuse@apache.org>
- Enforce a more reasonable limit on the number of included/excluded indexes
- Serialize vints rather than shorts in IndexSetSerializer
- Return Iterable from notExcluded() to avoid set creation
- Avoid redundant iteration in MessagingService#endpointsWithConnectionsOnVersionBelow()
patch by Caleb Rackliffe; reviewed by Marcus Eriksson for CASSANDRA-20888
Avoid audit batch event creation if audit for batch is not enabled (random UIID generation is not very cheap)
Avoid String.format in a potential hot path
Patch by Dmitry Konstantinov; reviewed by Štefan Miklošovič for CASSANDRA-20885
Co-authored-by: Štefan Miklošovič
As per CASSANDRA-20045, we want to prevent full repair against
disk full scenarios. Current protection exists only for incremental
repair. This change updates the config name to not be
incremental repair specific, using the Replace annotation.
patch by Himanshu Jindal; reviewed by David Capwell, Jaydeepkumar Chovatia for CASSANDRA-20045
Adds setNativeTransportMaxConcurrentConnectionsPerIp and
getNativeTransportMaxConcurrentConnectionsPerIp to StorageProxyMBean so
these methods can be used from JMX like the
*NativeTransportMaxConcurrentConnections methods.
patch by Andy Tolbert; reviewed by Chris Lohfink for CASSANDRA-20642
ninja: remove unused imports in CommandsForRanges to pass checkstyle
Patch by Dmitry Konstantinov; reviewed by Stefan Miklosovic, Caleb Rackliffe for CASSANDRA-20842
Avoid double lookup of the same DataPlacement in forNonLocalStrategyTokenRead and forNonLocalStrategyTokenWrite methods
Memorize hashCode value for ReplicationParams
Deduplicate ReplicationParams to use the same objects in DataPlacements and KeyspaceMetadata to use the fast == path in the equals
Do not search endpoints for a token in a typical write case twice (to identify pending endpoints)
Patch by Dmitry Konstantinov; reviewed by Stefan Miklosovic, Sam Tunnicliffe for CASSANDRA-20804
This also allows overriding complex settings as a JSON value and adds documentation about these overrides to conf/jvm-server.options
patch by Paulo Motta; reviewed by Stefan Miklosovic, David Capwell for CASSANDRA-20749
The map lookup (with an implicit auto-boxing) can be replaced with a plain arithmetic operation.
A unit test is used to ensure that the assumption about serialization version incrementing for new versions is still correct.
greaterThanOrEqual method is removed as unused.
Patch by Dmitry Konstantinov; reviewed by Stefan Miklosovic for CASSANDRA-20816
A flushing range is a combination of partial and full shards, for full shards we use write time captured values for count and total size of partition keys, for partial shards we iterate over keys
When we iterate over partial shards we count key bytes only instead of reading them and also try to skip tokens full parsing
Patch by Dmitry Konstantinov; reviewed by Branimir Lambov for CASSANDRA-20760
patch by Romain Hardouin; reviewed by Stefan Miklosovic, Dmitry Konstantinov for CASSANDRA-20494
Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
patch by Caleb Rackliffe; reviewed by Andres de la Peña for CASSANDRA-18112
Co-authored-by: Caleb Rackliffe <calebrackliffe@gmail.com>
Co-authored-by: Andres de la Peña <a.penya.garcia@gmail.com>
Sourcing cassandra-env.sh in bin/nodetool just to pick JMX_PORT from there
has undesirable side-effect of not being able to run nodetool when
MAX_HEAP_SIZE in cassandra-env.sh is uncommented.
patch by Stefan Miklosovic; reviewed by Dmitry Konstantinov, Michael Semb Wever for CASSANDRA-20745
Replace io.airlift:airline dependency with info.picocli:picocli across
all nodetool command implementations. This migration includes:
- Convert 160+ nodetool command classes from airline to picocli
- Add AbstractCommand base class for commands and utility methods
- Add NodetoolCommand top-level class for the cli utility
- Update all command classes to use @Command, @Option, and @Parameters
- Add plain text files to test command help output
- Add mock test classes for improved test coverage and parse validation
- Modify test infrastructure to work with picocli-based commands
- Add new layouts for cli formatting to preserve backwards compatibility
The migration maintains backward compatibility while providing improved
command-line parsing, better help system, and more robust argument
validation through picocli enhanced features.
patch by Maxim Muzafarov; reviewed by Caleb Rackliffe, Dmitry Konstantinov, Stefan Miklosovic for CASSANDRA-17445
patch by Yuqi Yan; reviewed by Brandon Williams, Maxwell Guo, Stefan Miklosovic for CASSANDRA-19552
Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
SAI predicate search currently has a bug that could result in missing rows due to a
concurrent flush during a query. The new test created in this PR shows the point of failure.
The problem is that we get the SSTable index references before getting the Memtable index
references. Note that we do it in the correct order in the ANN OF query path, but not
in the WHERE query path.
This commit updates the QueryView object to hold references to the appropriate Memtable indexes.
It also removes the problematic search methods from MemtableIndexManager to prevent future misuse.
patch by Michael Marshall; reviewed by Caleb Rackliffe and Ekaterina Dimitrova for CASSANDRA-20709
Updates SystemKeyspace.writePreparedStatement to accept a timestamp
associated with the Prepared creation time. Using this timestamp
will ensure that an INSERT into system.prepared_statements will
always precede the timestamp for the same Prepared in
SystemKeyspace.removePreparedStatement.
This is needed because Caffeine 2.9.2 may evict an entry as soon
as it is inserted if the maximum weight of the cache is exceeded
causing the DELETE to be executed before the INSERT.
Additionally, any clusters currently experiencing a leaky
system.prepared_statements table from this bug may struggle to
bounce into a version with this fix as
SystemKeyspace.loadPreparedPreparedStatements currently does
not paginate the query to system.prepared_statements, causing heap
OOMs. To fix this this patch adds pagination at 5000 rows and
aborts loading once the cache size is loaded. This should allow
nodes to come up and delete older prepared statements that may no
longer be used as the cache fills up (which should happen immediately).
This patch does not address the issue of Caffeine immediately evicting
a prepared statement, however it will prevent the
system.prepared_statements table from growing unbounded. For most users
this should be adequate, as the cache should only be filled when there
are erroneously many unique prepared statements. In such a case we can
expect that clients will constantly prepare statements regardless
of whether or not the cache is evicting statements.
patch by Andy Tolbert; reviewed by Berenguer Blasi and Caleb Rackliffe for CASSANDRA-19703
While we do not support Windows as such (at least on server), reviewers evaluated that this might be fixed
as the gains (Windows users using CQLSH to connect to Cassandra running on supported platforms) are justified.
patch by Brad Schoening; reviewed by Brandon Williams, Josh McKenzie for CASSANDRA-20478
As part of a broader effort to decouple java driver code from the
server code, this moves sstableloader to its own tools directory.
As sstableloader is also used as a library (CASSANDRA-10637), added
a new artifact 'cassandra-sstableloader' that will get deployed to
maven along with 'cassandra-all'.
While I expect this is likely a niche use case, this will allow users
to continue using BulkExport as a library.
Moves sstableloader-specific targets to its own build.xml in
tools/sstableloader/build.xml.
Also updates IDE project files and circleci to utilize new
sstableloader-specific targets.
patch by Andy Tolbert; reviewed by Stefan Miklosovic and Mick Semb Wever for CASSANDRA-20328
patch by Stefan Miklosovic; reviewed by Bernardo Botella Corbi for CASSANDRA-20563
Co-authored-by: Bernardo Botella Corbi <contacto@bernardobotella.com>
Memory-mapping is done in buffers of size less than 2GiB.
When these buffers aren't aligned to 4KiB and the trie-index file
spans many buffers then reading it results in going out of buffer
bounds.
This patch fixes it by making sure that the buffers are correctly
aligned.
patch by Szymon Miezal; reviewed by blambov and brandonwilliams for CASSANDRA-20351
Abe Ratnofsky