natas_hw
/
cassandra
mirror of https://github.com/apache/cassandra
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Increasing default for auto_repair.sstable_upper_threshold considering large Cassandra tables

This commit is contained in:
jaydeepkumar1984 2025-04-24 12:04:22 -07:00 committed by Chris Lohfink
parent 84fdf2efd9
commit 58d1cc9b1e
6 changed files with 12 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
CHANGES.txt
View File

@ -1,9 +1,12 @@
5.1 5.1
* Increasing default for auto_repair.sstable_upper_threshold considering large Cassandra tables & revert three lines removed from CHANGES.txt due to a merge mistake (CASSANDRA-20586)
* Fix token restrictions with MIN_TOKEN (CASSANDRO-20557) * Fix token restrictions with MIN_TOKEN (CASSANDRO-20557)
* Upgrade logback version to 1.5.18 and slf4j dependencies to 2.0.17 (CASSANDRA-20429) * Upgrade logback version to 1.5.18 and slf4j dependencies to 2.0.17 (CASSANDRA-20429)
* Switch memtable-related off-heap objects to Native Endian and Memory to Little Endian (CASSANDRA-20190) * Switch memtable-related off-heap objects to Native Endian and Memory to Little Endian (CASSANDRA-20190)
* Change SSTableSimpleScanner to use SSTableReader#openDataReaderForScan (CASSANDRA-20538) * Change SSTableSimpleScanner to use SSTableReader#openDataReaderForScan (CASSANDRA-20538)
* Automated Repair Inside Cassandra [CEP-37] (CASSANDRA-19918) * Automated Repair Inside Cassandra [CEP-37] (CASSANDRA-19918)
* Implement appender of slow queries to system_views.slow_queries table (CASSANDRA-13001)
* Add autocompletion in CQLSH for built-in functions (CASSANDRA-19631)
* Grant permission on keyspaces system_views and system_virtual_schema not possible (CASSANDRA-20171) * Grant permission on keyspaces system_views and system_virtual_schema not possible (CASSANDRA-20171)
* General Purpose Transactions (Accord) [CEP-15] (CASSANDRA-17092) * General Purpose Transactions (Accord) [CEP-15] (CASSANDRA-17092)
* Improve performance when getting writePlacementsAllSettled from ClusterMetadata (CASSANDRA-20526) * Improve performance when getting writePlacementsAllSettled from ClusterMetadata (CASSANDRA-20526)
@ -243,6 +246,7 @@ Merged from 4.1:
* Enforce CQL message size limit on multiframe messages (CASSANDRA-20052) * Enforce CQL message size limit on multiframe messages (CASSANDRA-20052)
* Fix race condition in DecayingEstimatedHistogramReservoir during rescale (CASSANDRA-19365) * Fix race condition in DecayingEstimatedHistogramReservoir during rescale (CASSANDRA-19365)
Merged from 4.0: Merged from 4.0:
* Updating a column with a new TTL but same expiration time is non-deterministic and causes repair mismatches. (CASSANDRA-20561)
* Avoid computing prepared statement size for unprepared batches (CASSANDRA-20556) * Avoid computing prepared statement size for unprepared batches (CASSANDRA-20556)
* Fix Dropwizard Meter causes timeouts when infrequently used (CASSANDRA-19332) * Fix Dropwizard Meter causes timeouts when infrequently used (CASSANDRA-19332)
* Update OWASP dependency checker to version 12.1.0 (CASSANDRA-20501) * Update OWASP dependency checker to version 12.1.0 (CASSANDRA-20501)

2
conf/cassandra.yaml
View File

@ -2810,7 +2810,7 @@ storage_compatibility_mode: NONE
# force_repair_new_node: false # force_repair_new_node: false
# # Threshold to skip repairing tables with too many SSTables. Defaults to 10,000 SSTables to avoid penalizing good # # Threshold to skip repairing tables with too many SSTables. Defaults to 10,000 SSTables to avoid penalizing good
# # tables. # # tables.
# sstable_upper_threshold: 10000 # sstable_upper_threshold: 50000
# # Maximum time allowed for repairing one table on a given node. If exceeded, the repair proceeds to the # # Maximum time allowed for repairing one table on a given node. If exceeded, the repair proceeds to the
# # next table. # # next table.
# table_max_repair_time: 6h # table_max_repair_time: 6h

2
conf/cassandra_latest.yaml
View File

@ -2499,7 +2499,7 @@ storage_compatibility_mode: NONE
# force_repair_new_node: false # force_repair_new_node: false
# # Threshold to skip repairing tables with too many SSTables. Defaults to 10,000 SSTables to avoid penalizing good # # Threshold to skip repairing tables with too many SSTables. Defaults to 10,000 SSTables to avoid penalizing good
# # tables. # # tables.
# sstable_upper_threshold: 10000 # sstable_upper_threshold: 50000
# # Maximum time allowed for repairing one table on a given node. If exceeded, the repair proceeds to the # # Maximum time allowed for repairing one table on a given node. If exceeded, the repair proceeds to the
# # next table. # # next table.
# table_max_repair_time: 6h # table_max_repair_time: 6h

6
doc/modules/cassandra/pages/managing/operating/auto_repair.adoc
View File

@ -207,7 +207,7 @@ node. Defaults to true and is only evaluated when allow_parallel_replica_repair
immediately after a restart. immediately after a restart.
| repair_session_timeout | 3h | Timeout for retrying stuck repair sessions. | repair_session_timeout | 3h | Timeout for retrying stuck repair sessions.
| force_repair_new_node | false | Force immediate repair on new nodes after they join the ring. | force_repair_new_node | false | Force immediate repair on new nodes after they join the ring.
| sstable_upper_threshold | 10000 | Threshold to skip repairing tables with too many SSTables. | sstable_upper_threshold | 50000 | Threshold to skip repairing tables with too many SSTables.
| table_max_repair_time | 6h | Maximum time allowed for repairing one table on a given node. If exceeded, the repair | table_max_repair_time | 6h | Maximum time allowed for repairing one table on a given node. If exceeded, the repair
proceeds to the next table. proceeds to the next table.
| ignore_dcs | [] | Avoid running repairs in specific data centers. By default, repairs run in all data centers. Specify | ignore_dcs | [] | Avoid running repairs in specific data centers. By default, repairs run in all data centers. Specify
@ -365,7 +365,7 @@ configuration for repair_type: full
min_repair_interval: 24h min_repair_interval: 24h
repair_by_keyspace: true repair_by_keyspace: true
number_of_repair_threads: 1 number_of_repair_threads: 1
sstable_upper_threshold: 10000 sstable_upper_threshold: 50000
table_max_repair_time: 6h table_max_repair_time: 6h
ignore_dcs: [] ignore_dcs: []
repair_primary_token_range_only: true repair_primary_token_range_only: true
@ -387,7 +387,7 @@ configuration for repair_type: incremental
min_repair_interval: 1h min_repair_interval: 1h
repair_by_keyspace: true repair_by_keyspace: true
number_of_repair_threads: 1 number_of_repair_threads: 1
sstable_upper_threshold: 10000 sstable_upper_threshold: 50000
table_max_repair_time: 6h table_max_repair_time: 6h
ignore_dcs: [] ignore_dcs: []
repair_primary_token_range_only: true repair_primary_token_range_only: true

2
src/java/org/apache/cassandra/repair/autorepair/AutoRepairConfig.java
View File

@ -452,7 +452,7 @@ public class AutoRepairConfig implements Serializable
opts.parallel_repair_percentage = 3; opts.parallel_repair_percentage = 3;
opts.allow_parallel_replica_repair = false; opts.allow_parallel_replica_repair = false;
opts.allow_parallel_replica_repair_across_schedules = true; opts.allow_parallel_replica_repair_across_schedules = true;
opts.sstable_upper_threshold = 10000; opts.sstable_upper_threshold = 50000;
opts.ignore_dcs = new HashSet<>(); opts.ignore_dcs = new HashSet<>();
opts.repair_primary_token_range_only = true; opts.repair_primary_token_range_only = true;
opts.force_repair_new_node = false; opts.force_repair_new_node = false;

4
test/unit/org/apache/cassandra/repair/autorepair/AutoRepairConfigTest.java
View File

@ -474,7 +474,7 @@ public class AutoRepairConfigTest extends CQLTester
assertEquals(Integer.valueOf(1), options.number_of_repair_threads); assertEquals(Integer.valueOf(1), options.number_of_repair_threads);
assertEquals(Integer.valueOf(3), options.parallel_repair_count); assertEquals(Integer.valueOf(3), options.parallel_repair_count);
assertEquals(Integer.valueOf(3), options.parallel_repair_percentage); assertEquals(Integer.valueOf(3), options.parallel_repair_percentage);
assertEquals(Integer.valueOf(10000), options.sstable_upper_threshold); assertEquals(Integer.valueOf(50000), options.sstable_upper_threshold);
assertEquals(new HashSet<>(), options.ignore_dcs); assertEquals(new HashSet<>(), options.ignore_dcs);
assertTrue(options.repair_primary_token_range_only); assertTrue(options.repair_primary_token_range_only);
assertFalse(options.force_repair_new_node); assertFalse(options.force_repair_new_node);
@ -495,7 +495,7 @@ public class AutoRepairConfigTest extends CQLTester
assertEquals(Integer.valueOf(1), config.global_settings.number_of_repair_threads); assertEquals(Integer.valueOf(1), config.global_settings.number_of_repair_threads);
assertEquals(Integer.valueOf(3), config.global_settings.parallel_repair_count); assertEquals(Integer.valueOf(3), config.global_settings.parallel_repair_count);
assertEquals(Integer.valueOf(3), config.global_settings.parallel_repair_percentage); assertEquals(Integer.valueOf(3), config.global_settings.parallel_repair_percentage);
assertEquals(Integer.valueOf(10000), config.global_settings.sstable_upper_threshold); assertEquals(Integer.valueOf(50000), config.global_settings.sstable_upper_threshold);
assertEquals(new HashSet<>(), config.global_settings.ignore_dcs); assertEquals(new HashSet<>(), config.global_settings.ignore_dcs);
assertTrue(config.global_settings.repair_primary_token_range_only); assertTrue(config.global_settings.repair_primary_token_range_only);
assertFalse(config.global_settings.force_repair_new_node); assertFalse(config.global_settings.force_repair_new_node);