Add BE offsets detection on Summary loading.
Add test SSTables in an old format with BE offsets in Summary component to LegacySSTableTest.
Patch by Dmitry Konstantinov; reviewed by Branimir Lambov, Michael Semb Wever for CASSANDRA-20190
Also add test method testVerifyOldTupleSSTables for frozen tuples and dropping of them.
The test data also adds different versions of me format sstables from C* version 3.0.25 and 3.11.11 as they do (unfortunately) differ. Test sstable files have had their sequence generation ids changed to represent the server version used to create them, while this shouldn't be necessary it is for debug purposes when sstable formats have changed between versions.
patch by Roxana Neophytou, Mick Semb Wever; reviewed by Dmitry Konstantinov for CASSANDRA-20485
Co-authored-by: mck <mck@apache.org>
The approach is documented in the included UnifiedCompactionStrategy.md.
Also included:
- Generalize prettyPrint to prettyPrintBinary (with 2^10 multiplier,
e.g. MiB) and prettyPrintDecimal (with 1000 multiplier, e.g. ms)
that cover the full range of double, and add a parseHumanReadable
method that can read both.
- CASSANDRA-18123: Fix invalid reuse of metadata collector during flushing
- Fix invalid token range splitting with non-unit weights
- Add flushSizeOnDisk metric
- Add ability to change compaction default in YAML
patch by Alex Sorokoumov, Branimir Lambov, Dimitar Dimitrov and Stefania Alborghetti
reviewed by Alex Sorokoumov, Jaroslaw Grabowski and Maxwell Guo for CASSANDRA-18397
Documentation in the included BTIFormat.md.
patch by Branimir Lambov and Jacek Lewandowski; reviewed by Caleb Rackliffe and Maxwell Guo for CASSANDRA-18398
This patch improves the following things:
1. SSTable metadata will store a covered slice instead of min/max clusterings. The difference is that for slices there is available the type of a bound rather than just a clustering. In particular it will provide the information whether the lower and upper bound of an sstable is opened or closed.
2. SSTable metadata will store a flag whether the SSTable contains any partition level deletions or not
3. The above two changes required to introduce a new major format for SSTables - oa
4. Single partition read command makes use of the above changes. In particular an sstable can be skipped when it does not intersect with the column filter, does not have partition level deletions and does not have statics; In case there are partition level deletions, but the other conditions are satisfied, only the partition header needs to be accessed (tests attached)
5. Skipping SSTables assuming those three conditions are satisfied has been implemented also for partition range queries (tests attached). Also added minor separate statistics to record the number of accessed sstables in partition reads because now not all of them need to be accessed.
6. Artificial lower bound marker is now an object on its own and is not implemented as a special case of range tombstone bound.
7. Extended the lower bound optimization usage due the 1 and 2
8. Do not initialize iterator just to get a cached partition and associated columns index. The purpose of using lower bound optimization was to avoid opening an iterator of an sstable if possible.
9. Add key range to stats metadata
[f369595b1c] Add fields to sstable version and placeholders in stats serializer
[f5c3f772e2] Add hasKeyRange and hasLegacyMinMax
[3cde51f4e1] Add partition level deletion presence marker to sstable stats
[67b2ee2152] Extract AbstractTypeSerializer
[c77b475d6c] Refactor slices intersection checking
[ceb5af3a38] Store min and max clustering as a slice in stats metadata as and improved min/max
[d1f8973929] Implement MetadataCollectorBench
[335369da84] Apply partition level deletion presence marker optimizations to single partition read command
[2497a009b9] Lower bound optimization - add slices and isReverseOrder fields to UnfilteredRowIteratorWithLowerBound
[e32ee31177] Lower bound optimization - Replace usage of RangeTombstoneMarker as a lower bound with ArtificialBoundMarker
[e213e712c4] Lower bound optimization - improve usage of lower bound optimization
[c4f93006b1] Apply read path improvements to partition range queries
[5fa462266c] Add key range to StatsMetadata
[79a7339ed4] Use key range from stats if possible
[266ed2749b] Added new sstables for LegacySSTableTest
patch by Jacek Lewandowski; reviewed by Branimir Lambov and C. Scott Andreas for CASSANDRA-18134
Co-authored-by: Branimir Lambov <blambov>
Co-authored-by: Sylvain Lebresne <pcmanus>
Co-authored-by: Jacek Lewandowski <jacek-lewandowski>
Co-authored-by: Jakub Zytka <jakubzytka>
patch by Jakub Zytka and Jacek Lewandowski; reviewed by Branimir Lambov and Benjamin Lerer for CASSANDRA-16619
Add originating host ID to sstable metadata
- md, me sstables
- add missing test cases in MetadataSerializerTest
Co-authored-by: Jakub Zytka <jakub.zytka@datastax.com>
Co-authored-by: Jacek Lewandowski <jacek.lewandowski@datastax.com>
Alternative solution for CASSANDRA-15086, which allows the illegal cells to
be read from disk and deserialized as normal so as not to interfere with
tracking bytes read during reverse iteration of indexed partitions.
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-15178
Patch by Blake Eggleston, Benedict Elliott Smith, Marcus Eriksson, Alex Petrov, Ariel Weisberg; Reviewed by Blake Eggleston, Marcus Eriksson, Benedict Elliott Smith, Alex Petrov, Ariel Weisberg for CASSANDRA-14404
Co-authored-by: Blake Eggleston <bdeggleston@gmail.com>
Co-authored-by: Benedict Elliott Smith <benedict@apache.org>
Co-authored-by: Marcus Eriksson <marcuse@apache.org>
Co-authored-by: Alex Petrov <oleksandr.petrov@gmail.com>
To improve clarity of control flow, all iterator transformations
are applied via a single class that manages an explicit stack of
named transformation objects.
patch by benedict; reviewed by branimir for CASSANDRA-9975
Dmitry Konstantinov