- Adds jbellis/jvector (1.0.2) library for DiskANN based indexes on floating point vectors
- Adds ORDER BY ANN OF capability to do ANN search and order the results by score
patch by Mike Adamson; reviewed by Andrés de la Peña, Jonathon Ellis for CASSANDRA-18715
Co-authored-by Jonathon Ellis jbellis@gmail.com
Co-authored-by Zhao Yang zhaoyangsingapore@gmail.com
The approach is documented in the included UnifiedCompactionStrategy.md.
Also included:
- Generalize prettyPrint to prettyPrintBinary (with 2^10 multiplier,
e.g. MiB) and prettyPrintDecimal (with 1000 multiplier, e.g. ms)
that cover the full range of double, and add a parseHumanReadable
method that can read both.
- CASSANDRA-18123: Fix invalid reuse of metadata collector during flushing
- Fix invalid token range splitting with non-unit weights
- Add flushSizeOnDisk metric
- Add ability to change compaction default in YAML
patch by Alex Sorokoumov, Branimir Lambov, Dimitar Dimitrov and Stefania Alborghetti
reviewed by Alex Sorokoumov, Jaroslaw Grabowski and Maxwell Guo for CASSANDRA-18397
Documentation in the included BTIFormat.md.
patch by Branimir Lambov and Jacek Lewandowski; reviewed by Caleb Rackliffe and Maxwell Guo for CASSANDRA-18398
– update MessagingService (remove the 4.1 version that was only a placeholder in test code bc serialisation test files were previously incorrectly serialising random UUIds for TimeUUIDs)
– update upgrade jvm-dtests
– remove older serialization binary files we are no longer testing against
patch by Mick Semb Wever; reviewed by Brandon Williams for CASSANDRA-17973
This patch improves the following things:
1. SSTable metadata will store a covered slice instead of min/max clusterings. The difference is that for slices there is available the type of a bound rather than just a clustering. In particular it will provide the information whether the lower and upper bound of an sstable is opened or closed.
2. SSTable metadata will store a flag whether the SSTable contains any partition level deletions or not
3. The above two changes required to introduce a new major format for SSTables - oa
4. Single partition read command makes use of the above changes. In particular an sstable can be skipped when it does not intersect with the column filter, does not have partition level deletions and does not have statics; In case there are partition level deletions, but the other conditions are satisfied, only the partition header needs to be accessed (tests attached)
5. Skipping SSTables assuming those three conditions are satisfied has been implemented also for partition range queries (tests attached). Also added minor separate statistics to record the number of accessed sstables in partition reads because now not all of them need to be accessed.
6. Artificial lower bound marker is now an object on its own and is not implemented as a special case of range tombstone bound.
7. Extended the lower bound optimization usage due the 1 and 2
8. Do not initialize iterator just to get a cached partition and associated columns index. The purpose of using lower bound optimization was to avoid opening an iterator of an sstable if possible.
9. Add key range to stats metadata
[f369595b1c] Add fields to sstable version and placeholders in stats serializer
[f5c3f772e2] Add hasKeyRange and hasLegacyMinMax
[3cde51f4e1] Add partition level deletion presence marker to sstable stats
[67b2ee2152] Extract AbstractTypeSerializer
[c77b475d6c] Refactor slices intersection checking
[ceb5af3a38] Store min and max clustering as a slice in stats metadata as and improved min/max
[d1f8973929] Implement MetadataCollectorBench
[335369da84] Apply partition level deletion presence marker optimizations to single partition read command
[2497a009b9] Lower bound optimization - add slices and isReverseOrder fields to UnfilteredRowIteratorWithLowerBound
[e32ee31177] Lower bound optimization - Replace usage of RangeTombstoneMarker as a lower bound with ArtificialBoundMarker
[e213e712c4] Lower bound optimization - improve usage of lower bound optimization
[c4f93006b1] Apply read path improvements to partition range queries
[5fa462266c] Add key range to StatsMetadata
[79a7339ed4] Use key range from stats if possible
[266ed2749b] Added new sstables for LegacySSTableTest
patch by Jacek Lewandowski; reviewed by Branimir Lambov and C. Scott Andreas for CASSANDRA-18134
Co-authored-by: Branimir Lambov <blambov>
Co-authored-by: Sylvain Lebresne <pcmanus>
Co-authored-by: Jacek Lewandowski <jacek-lewandowski>
Co-authored-by: Jakub Zytka <jakubzytka>
This work encompasses a range of improvements to Paxos, summarised as:
- The introduction of Paxos Repair for ensuring consistency during range movements
- The reduction of network round-trips by a factor of 2x for reads and writes
patch by Benedict Elliott Smith and Blake Eggleston; reviewed by Alex Petrov, Aleksey Yeschenko and Sam Tunnicliffe for CASSANDRA-17164
patch by Jakub Zytka and Jacek Lewandowski; reviewed by Branimir Lambov and Benjamin Lerer for CASSANDRA-16619
Add originating host ID to sstable metadata
- md, me sstables
- add missing test cases in MetadataSerializerTest
Co-authored-by: Jakub Zytka <jakub.zytka@datastax.com>
Co-authored-by: Jacek Lewandowski <jacek.lewandowski@datastax.com>
This reverts the changes made to the native protocol V5 wire format in
CASSANDRA-13304 and subsequent follow up JIRAs (CASSANDRA-15556 &
CASSANDRA-14716). The framing format has been re-implemented based on the
internode messaging format developed in CASSANDRA-15066.
OPTIONS and STARTUP messages are unframed (i.e. identical to the V4 format).
After sending its response to a STARTUP message, the server modifies the
pipeline and all further messages are sent (and should be expected by the
client to be received) in the format determined by the protocol version of
the STARTUP message.
Patch by Sam Tunnicliffe; reviewed by Alex Petrov and Caleb Rackliffe
for CASSANDRA-15299
* initiate multiple buffer pool for different lifespan and usages
- Chunk Cache Buffer Pool - conf.file_cache_size_in_mb=512mb
- Networking Buffer Pool - conf.temporary_cache_size_in_mb=128mb
* Add overflowSize and usedSize to buffer pool metrics
* re-circulate buffer pool Chunk for ChunkCache whenever it has free space, even thoughput it may not be able to allocate due to fragmentation
patch by Zhao Yang; reviewed by Caleb Rackliffe and Aleksey Yeschenko for CASSANDRA-15229
Alternative solution for CASSANDRA-15086, which allows the illegal cells to
be read from disk and deserialized as normal so as not to interfere with
tracking bytes read during reverse iteration of indexed partitions.
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-15178
patch by Aleksey Yeschenko and Benedict Elliott Smith; reviewed by Alex
Petrov, Aleksey Yeschenko, and Benedict Elliott Smith for CASSANDRA-15066
Co-authored-by: Aleksey Yeschenko <aleksey@apache.org>
Co-authored-by: Benedict Elliott Smith <benedict@apache.org>
Stefan Miklosovic