Commit Graph

18864 Commits

Author SHA1 Message Date
Mike Adamson 3928f2992f
Provide truncate task for SAI
If a table is truncated during the initial build of an index there is a chance that the index build will get interrupted and it won't get marked queryable. This patch provides a truncate task for SAI that marks the index queryable during truncation.

 patch by Mike Adamson; reviewed by Caleb Rackliffe, Michael Semb Wever for CASSANDRA-19032
2023-11-29 13:52:46 +01:00
Stefan Miklosovic 3396ce9f08
Merge branch 'cassandra-4.1' into cassandra-5.0 2023-11-29 12:14:27 +01:00
Stefan Miklosovic c8d6182148
Merge branch 'cassandra-4.0' into cassandra-4.1 2023-11-29 12:12:56 +01:00
Stefan Miklosovic 8dc32c5dd7
Merge branch 'cassandra-3.11' into cassandra-4.0 2023-11-29 12:12:22 +01:00
Stefan Miklosovic 6d7cd61412
Merge branch 'cassandra-3.0' into cassandra-3.11 2023-11-29 12:11:29 +01:00
Stefan Miklosovic c1b12058e7
Do not set RPC_READY to false on transports shutdown in order to not fail counter updates for deployments with coordinator and storage nodes with transports turned off
This is the follow-up commit of CASSANDRA-18935 where we set RPC_READY to false
when transports were shut down in runtime. The problem is that the current logic in StorageProxy.findSuitableEndpoint method,
used for the selection of a leader for counter mutations, is filtering out all endpoints which do not have RPC_READY
set to true. Hence, if there is a deployment of a coordinator node (not joining a ring) and storage nodes which
have transports turned off (e.g. for security reasons), then a coordinator node will select no endpoint as a
counter mutation leader which renders counter mutations impossible.

This change just reverts the original fix which was setting RPC_READY to false when transports were shut down in runtime
(e.g. by nodetool disablebinary).

For trunk (to be 5.1 at time of writing this), there is already TCM in place and the correct fix yet to be implemented
is to decouple from checking RCP_READY state and base it e.g. on JOINED state from TCM perspective. Please follow
CASSANDRA-19103 where this will be addressed.

patch by Stefan Miklosovic; reviewed by Brandon Williams for CASSANDRA-18935
2023-11-29 12:09:59 +01:00
Amit Pawar 3259bea533 Enable Direct-IO feature for CommitLog files using Java native API's.
Patch by Amit Pawar and Jacek Lewandowski; reviewed by Branimir Lambov and Maxwell Guo for CASSANDRA-18464

Co-authored-by: Amit Pawar <Amit.Pawar@amd.com>
Co-authored-by: Jacek Lewandowski <lewandowski.jacek@gmail.com>
2023-11-29 11:29:23 +01:00
Jacek Lewandowski 3cdf71defe Merge branch 'cassandra-4.1' into cassandra-5.0
* cassandra-4.1:
  Fix filtering system ks sstables for relocation on startup
2023-11-28 18:50:18 +01:00
Jacek Lewandowski 78dd963481 Merge branch 'cassandra-4.0' into cassandra-4.1
* cassandra-4.0:
  Fix filtering system ks sstables for relocation on startup
2023-11-28 14:21:16 +01:00
Jacek Lewandowski dece96f21d Fix filtering system ks sstables for relocation on startup
Patch by Jacek Lewandowski; reviewed by Benjamin Lerer for CASSANDRA-18963
2023-11-28 14:20:26 +01:00
Mike Adamson a9e6ed3787
Fix broken indexing tests when using SAI
- This fixes a number of broken tests when the default index is set to SAI
 - Composite partition indexes were being filtered prior to row filtering in the
   index searcher resulting in incorrect results
 - Static and non-static index intersection was failing because static primary keys
   were not comparing correctly against non-static primary keys

 patch by Mike Adamson; reviewed by Andres de la Peña, Michael Semb Wever for CASSANDRA-19034
2023-11-26 12:27:23 +01:00
Mike Adamson 0e42b77c97
Improve code model around IndexContext
- Replace IndexContext with IndexTermType and IndexDefinition
 - Move index specific managers, factories and metrics to StorageAttachedIndex
 - Refactor Expression to explicitly define indexed and unindexed expressions

 patch by Mike Adamson; reviewed by Andres de la Peña, Caleb Rackliffe for CASSANDRA-18166
2023-11-25 09:53:40 +01:00
Ethan Brown 9c6e671434
Fixed bug where UnifiedCompactionTask constructor was calling the wrong base constructor of CompactionTask
patch by Ethan Brown; reviewed by Branimir Lambov, Stefan Miklosovic for CASSANDRA-18757
2023-11-23 16:19:10 +01:00
Andrés de la Peña c26f2a3d69 Fix SAI unindexed contexts not considering CONTAINS KEY
patch by Andrés de la Peña; reviewed by Maxwell Guo and Michael Semb Wever for CASSANDRA-19040
2023-11-23 11:51:52 +00:00
Andrés de la Peña e8fb4b2f63 Ensure that empty SAI column indexes do not fail on validation after full-SSTable streaming
patch by Andrés de la Peña; reviewed by Caleb Rackliffe for CASSANDRA-19017

Co-authored-by: Andrés de la Peña <a.penya.garcia@gmail.com>
Co-authored-by: Caleb Rackliffe <calebrackliffe@gmail.com>
2023-11-22 15:13:06 +00:00
Andrés de la Peña 91a242fe00 SAI in-memory index should check max term size
patch by Zhao Yang and Andrés de la Peña; reviewed by Zhao Yang for CASSANDRA-18926

Co-authored-by: Zhao Yang <zhaoyangsingapore@gmail.com>
Co-authored-by: Andrés de la Peña <a.penya.garcia@gmail.com>
2023-11-20 15:59:55 +00:00
Bereng 6cac24f581 Merge branch 'cassandra-4.0' into cassandra-4.1
* cassandra-4.0:
  Test failure: org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest
2023-11-20 07:36:52 +01:00
Bereng 3ba6de70c7 Test failure: org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest
patch by Berenguer Blasi; reviewed by Andres de la Peña for CASSANDRA-18948
2023-11-20 07:34:57 +01:00
Stefan Miklosovic e1355d7238
Set default disk_access_mode to mmap_index_only
patch by Stefan Miklosovic; reviewed by Paulo Motta, Caleb Rackliffe, Brandon Williams for CASSANDRA-19021
2023-11-15 22:14:21 +01:00
Ethan Brown fb806d51e3
Add UCS sstable_growth and min_sstable_size options
patch by Ethan Brown; reviewed by Branimir Lambov, Stefan Miklosovic for CASSANDRA-18945
2023-11-14 21:41:07 +01:00
Stefan Miklosovic bd4e7d7824
Merge branch 'cassandra-4.1' into cassandra-5.0 2023-11-13 15:31:17 +01:00
Stefan Miklosovic afb422ddc0
Merge branch 'cassandra-4.0' into cassandra-4.1 2023-11-13 15:20:35 +01:00
Abe Ratnofsky fe1be800b4
Remove completed coordinator sessions
patch by Abe Ratnofsky; reviewed by Caleb Rackliffe, Marcus Eriksson for CASSANDRA-18903
2023-11-13 15:17:47 +01:00
Jacek Lewandowski 4d61359c21 CASSANDRA-18785: Add support for Sonar analysis
Patch by Jacek Lewandowski; reviewed by Brandon Williams, Maxim Muzafarov, Michael Semb Wever, Stefan Miklosovic for CASSANDRA-18785
2023-11-13 14:48:28 +01:00
Bereng 0e4c2f4bef Default to nb instead of nc for sstable formats
patch by Berenguer Blasi; reviewed by Francisco Guerrero, Jacek Lewandowski, Michael Semb Wever for CASSANDRA-19010
2023-11-13 09:26:11 +01:00
Stefan Miklosovic a5299bb453
Merge branch 'cassandra-4.0' into cassandra-4.1 2023-11-10 11:24:44 +01:00
Isaac Reath 26a7d574f0
Make StartupConnectivityChecker only run a connectivity check if there are no nodes which are running a version prior to Cassandra 4
patch by Isaac Reath; reviwed by Paulo Motta, Stefan Miklosovic for CASSANDRA-18968
2023-11-10 11:23:23 +01:00
Jacek Lewandowski 7a2bfdc56d Fix incorrect seeking through the sstable iterator by IndexState
Patch by Jacek Lewandowski; reviewed by Alex Petrov and Maxim Muzafarov for CASSANDRA-18932
2023-11-09 16:08:59 +01:00
Stefan Miklosovic 5b3c36b28a
Merge branch 'cassandra-4.1' into cassandra-5.0 2023-11-08 14:29:48 +01:00
David Capwell dca76145c2 Add retries to IR messages
patch by David Capwell; reviewed by Blake Eggleston for CASSANDRA-18962
2023-11-01 15:12:49 -07:00
Jacek Lewandowski f2c46cbf3b Merge branch 'cassandra-4.1' into cassandra-5.0
* cassandra-4.1:
  Retrieve keyspaces metadata and schema version concistently in DescribeStatement
2023-10-31 11:15:45 +01:00
Jacek Lewandowski 9ad5ec9664 Merge branch 'cassandra-4.0' into cassandra-4.1
* cassandra-4.0:
  Retrieve keyspaces metadata and schema version concistently in DescribeStatement
2023-10-31 11:15:44 +01:00
Jacek Lewandowski 86e07595f7 Retrieve keyspaces metadata and schema version concistently in DescribeStatement
The fix makes the DescribeStatement to wait for the in-progress schema transformations to finish before returning the first page. This way, the metadata and schema version encoded in the result set metadata are guaranteed to be consistent.

Patch by Jacek Lewandowski; reviewed by Benjamin Lerer, Ekaterina Dimitrova for CASSANDRA-18921
2023-10-31 11:15:43 +01:00
David Capwell 69f5df0b00 Add metrics and logging to repair retries
patch by David Capwell; reviewed by Caleb Rackliffe, Maxim Muzafarov for CASSANDRA-18952
2023-10-30 14:47:26 -07:00
Stefan Miklosovic 4ecff92404
Remove deprecated code in Cassandra 1.x and 2.x
patch by Stefan Miklosovic; reviewed by Brandon Williams for CASSANDRA-18959
2023-10-30 21:54:55 +01:00
Caleb Rackliffe 256e39fc62 ClientRequestSize metrics should not treat CONTAINS restrictions as being equality-based
patch by Caleb Rackliffe; reviewed by Benjamin Lerer for CASSANDRA-18896
2023-10-30 14:18:52 -05:00
Mike Adamson c4d11c4372 Fix VectorUpdateDeleteTest for JDK 17
Removed use of reflection and directly set
  relevant property to avoid jdk 17 errors

patch by Mike Adamson; reviewed by Stefan Miklosovic, Michael Semb Wever and Andrés de la Peña for CASSANDRA-18715
2023-10-30 18:57:54 +00:00
Mike Adamson 949b760f55
Add support for a vector search index in SAI
- Adds jbellis/jvector (1.0.2) library for DiskANN based indexes on floating point vectors
- Adds ORDER BY ANN OF capability to do ANN search and order the results by score

 patch by Mike Adamson; reviewed by Andrés de la Peña, Jonathon Ellis for CASSANDRA-18715

Co-authored-by Jonathon Ellis jbellis@gmail.com
Co-authored-by Zhao Yang zhaoyangsingapore@gmail.com
2023-10-29 12:24:00 +01:00
Stefan Miklosovic b59b832eba
Remove crc_check_chance from CompressionParams
patch by Stefan Miklosovic; reviewed by Maxwell Guo, Jacek Lewandowski, Branimir Lambov for CASSANDRA-18872
2023-10-26 22:38:21 +02:00
Aleksey Yeschenko 7d54a62ccc Fix schema loading of UDTs inside vectors inside UDTs
patch by Aleksey Yeschenko; reviewed by Marcus Eriksson for
CASSANDRA-18964
2023-10-26 17:44:10 +01:00
maxwellguo 0c7410a9da
Fix nodetool tablehistograms output to avoid printing repeated information and ensure at most two arguments
patch by Maxwell Guo; reviewed by Stefan Miklosovic and Brandon Williams for CASSANDRA-18955
2023-10-25 08:38:13 +02:00
Stefan Miklosovic 9ffb838d03
Merge branch 'cassandra-4.1' into cassandra-5.0 2023-10-24 16:32:37 +02:00
Stefan Miklosovic d6bf4fd98f
Merge branch 'cassandra-4.0' into cassandra-4.1 2023-10-24 16:26:07 +02:00
Stefan Miklosovic d6fcca95b4
Merge branch 'cassandra-3.11' into cassandra-4.0 2023-10-24 16:20:31 +02:00
Stefan Miklosovic 3987160bb2
Merge branch 'cassandra-3.0' into cassandra-3.11 2023-10-24 16:14:49 +02:00
Stefan Miklosovic b51ee83a29
Fix nodetool enable/disablebinary to correctly set rpc readiness in gossip
patch by Stefan Miklosovic; reviewed by Brandon Williams for CASSANDRA-18935
2023-10-24 16:12:37 +02:00
Maxim Muzafarov 07df26778b Change the checksum algorithm SAI-related files use from CRC32 to CRC32C
patch by Maxim Muzafarov; reviewed by Caleb Rackliffe and Zhao Yang for CASSANDRA-18836
2023-10-20 14:46:49 -05:00
Francisco Guerrero 8c89e2adb7 Correct comment for nc SSTable format
Patch by frankgh; reviewd by brandonwilliams and bereng for
CASSANDRA-18933
2023-10-19 12:27:57 -05:00
Jacek Lewandowski 57f494d8d4 Merge branch 'cassandra-4.1' into cassandra-5.0
* cassandra-4.1:
  Fixed the inconsistency between distributedKeyspaces and distributedAndLocalKeyspaces
2023-10-19 12:56:10 +02:00
Jacek Lewandowski ac71d0f56e Fixed the inconsistency between distributedKeyspaces and distributedAndLocalKeyspaces
Patch by Jacek Lewandowski; reviewed by Benjamin Lerer, Berenguer Blasi, Ekaterina Dimitrova, Jeremiah Jordan for CASSANDRA-18747
2023-10-19 12:56:09 +02:00