mirror of https://github.com/apache/cassandra
Add docs about configuration framework added in CASSANDRA-15234
patch by Ekaterina Dimitrova; reviewed by Michael Semb Wever and Caleb Rackliffe for CASSANDRA-17246
This commit is contained in:
Ekaterina Dimitrova
parent
34a7eba4b8
commit
d67be0def4
9
NEWS.txt
9
NEWS.txt
|
|
@ -116,6 +116,15 @@ New features
|
|||
|
||||
Upgrading
|
||||
---------
|
||||
- We added new JMX methods `setStreamThroughputMbitPerSec`, `getStreamThroughputMbitPerSec`, `setInterDCStreamThroughputMbitPerSec`,
|
||||
`getInterDCStreamThroughputMbitPerSec` to the JMX MBean `org.apache.cassandra.db:type=StorageService`. They replace the now
|
||||
deprecated methods `setStreamThroughputMbPerSec`, `getStreamThroughputMbPerSec`, `setInterDCStreamThroughputMbPerSec`, and
|
||||
`getInterDCStreamThroughputMbPerSec`, which will be removed in a future major release.
|
||||
- The config property `repair_session_space_in_mb` was wrongly advertised in previous versions that it should be set in
|
||||
megabytes when it is interpreted internally in mebibytes. To reduce the confusion we added two new JMX methods
|
||||
`setRepairSessionSpaceInMebibytes(int sizeInMebibytes)` and `getRepairSessionSpaceInMebibytes`. They replace the now
|
||||
deprecated methods `setRepairSessionSpaceInMegabytes(int sizeInMegabytes)` and `getRepairSessionSpaceInMegabytes`, which
|
||||
will be removed in a future major release.
|
||||
- There is a new cassandra.yaml version 2. Units suffixes should be provided for all rates(B/s|MiB/s|KiB/s|MiB/s),
|
||||
memory (KiB|MiB|GiB|B) and duration(d|h|s|ms|us|µs|ns|m)
|
||||
parameters. (CASSANDRA-15234)
|
||||
|
|
|
|||
|
|
@ -18,9 +18,9 @@ following properties on `cassandra.yaml`:
|
|||
|
||||
* `dynamic_snitch`: whether the dynamic snitch should be enabled or
|
||||
disabled.
|
||||
* `dynamic_snitch_update_interval_in_ms`: controls how often to perform
|
||||
* `dynamic_snitch_update_interval`: 100ms, controls how often to perform
|
||||
the more expensive part of host score calculation.
|
||||
* `dynamic_snitch_reset_interval_in_ms`: if set greater than zero, this
|
||||
* `dynamic_snitch_reset_interval`: 10m, if set greater than zero, this
|
||||
will allow 'pinning' of replicas to hosts in order to increase cache
|
||||
capacity.
|
||||
* `dynamic_snitch_badness_threshold:`: The badness threshold will
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ will be applied to memtables.
|
|||
|
||||
All mutations write optimized by storing in commitlog segments, reducing
|
||||
the number of seeks needed to write to disk. Commitlog Segments are
|
||||
limited by the `commitlog_segment_size_in_mb` option, once the size is
|
||||
limited by the `commitlog_segment_size` option, once the size is
|
||||
reached, a new commitlog segment is created. Commitlog segments can be
|
||||
archived, deleted, or recycled once all its data has been flushed to
|
||||
SSTables. Commitlog segments are truncated when Cassandra has written
|
||||
|
|
@ -19,16 +19,16 @@ data older than a certain point to the SSTables. Running "nodetool
|
|||
drain" before stopping Cassandra will write everything in the memtables
|
||||
to SSTables and remove the need to sync with the commitlogs on startup.
|
||||
|
||||
* `commitlog_segment_size_in_mb`: The default size is 32, which is
|
||||
* `commitlog_segment_size`: The default size is 32MiB, which is
|
||||
almost always fine, but if you are archiving commitlog segments (see
|
||||
commitlog_archiving.properties), then you probably want a finer
|
||||
granularity of archiving; 8 or 16 MB is reasonable. Max mutation size is
|
||||
also configurable via `max_mutation_size_in_kb` setting in `cassandra.yaml`.
|
||||
The default is half the size `commitlog_segment_size_in_mb * 1024`.
|
||||
granularity of archiving; 8 or 16 MiB is reasonable. `commitlog_segment_size`
|
||||
also determines the default value of `max_mutation_size` in cassandra.yaml.
|
||||
By default, max_mutation_size is half the size of `commitlog_segment_size`.
|
||||
|
||||
**NOTE: If `max_mutation_size_in_kb` is set explicitly then
|
||||
`commitlog_segment_size_in_mb` must be set to at least twice the size of
|
||||
`max_mutation_size_in_kb / 1024`**.
|
||||
**NOTE: If `max_mutation_size` is set explicitly then
|
||||
`commitlog_segment_size` must be set to at least twice the size of
|
||||
`max_mutation_size`**.
|
||||
|
||||
Commitlogs are an append only log of all mutations local to a Cassandra
|
||||
node. Any data written to Cassandra will first be written to a commit
|
||||
|
|
@ -47,11 +47,11 @@ for the same reason.
|
|||
- `commitlog_sync_batch_window_in_ms`: Time to wait between "batch"
|
||||
fsyncs _Default Value:_ 2
|
||||
** `periodic`: In periodic mode, writes are immediately ack'ed, and the
|
||||
CommitLog is simply synced every "commitlog_sync_period_in_ms"
|
||||
CommitLog is simply synced every "commitlog_sync_period"
|
||||
milliseconds.
|
||||
+
|
||||
- `commitlog_sync_period_in_ms`: Time to wait between "periodic" fsyncs
|
||||
_Default Value:_ 10000
|
||||
- `commitlog_sync_period`: Time to wait between "periodic" fsyncs
|
||||
_Default Value:_ 10000ms
|
||||
|
||||
_Default Value:_ batch
|
||||
|
||||
|
|
@ -79,7 +79,7 @@ Deflate and Zstd compressors are supported.
|
|||
# parameters:
|
||||
----
|
||||
|
||||
* `commitlog_total_space_in_mb`: Total space to use for commit logs on
|
||||
* `commitlog_total_space`: Total space to use for commit logs on
|
||||
disk.
|
||||
|
||||
If space gets above this value, Cassandra will flush every dirty CF in
|
||||
|
|
@ -89,7 +89,7 @@ tend to cause more flush activity on less-active columnfamilies.
|
|||
The default value is the smaller of 8192, and 1/4 of the total space of
|
||||
the commitlog volume.
|
||||
|
||||
_Default Value:_ 8192
|
||||
_Default Value:_ 8192MiB
|
||||
|
||||
== Memtables
|
||||
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ By default, the use of user-defined functions is disabled by default for
|
|||
security concerns (even when enabled, the execution of user-defined
|
||||
functions is sandboxed and a "rogue" function should not be allowed to
|
||||
do evil, but no sandbox is perfect so using user-defined functions is
|
||||
opt-in). See the `enable_user_defined_functions` in `cassandra.yaml` to
|
||||
opt-in). See the `user_defined_functions_enabled` in `cassandra.yaml` to
|
||||
enable them.
|
||||
====
|
||||
|
||||
|
|
|
|||
|
|
@ -107,10 +107,10 @@ Cassandra isn't optimized for large file or BLOB storage and a single
|
|||
storing small blobs (less than single digit MB) should not be a problem,
|
||||
but it is advised to manually split large blobs into smaller chunks.
|
||||
|
||||
Please note in particular that by default, any value greater than 16MB
|
||||
will be rejected by Cassandra due the `max_mutation_size_in_kb`
|
||||
Please note in particular that by default, any value greater than 16MiB
|
||||
will be rejected by Cassandra due the `max_mutation_size`
|
||||
configuration of the `cassandra-yaml` file (which default to half of
|
||||
`commitlog_segment_size_in_mb`, which itself default to 32MB).
|
||||
`commitlog_segment_size`, which itself default to 32MiB).
|
||||
|
||||
[[nodetool-connection-refused]]
|
||||
== Nodetool says "Connection refused to host: 127.0.1.1" for any remote host. What gives?
|
||||
|
|
|
|||
|
|
@ -0,0 +1,192 @@
|
|||
= Liberating cassandra.yaml Parameters' Names from Their Units
|
||||
|
||||
== Objective
|
||||
|
||||
Three big things happened as part of https://issues.apache.org/jira/browse/CASSANDRA-15234[CASSANDRA-15234]:
|
||||
|
||||
1) Renaming of parameters in `cassandra.yaml` to follow the form `noun_verb`.
|
||||
|
||||
2) Liberating `cassandra.yaml` parameters from their units (DataStorage, DataRate and Duration) and introducing temporary smallest accepted unit per parameter (only for DataStorage and Duration ones)
|
||||
|
||||
3) Backward compatibility framework to support the old names and lack of units support until at least the next major release.
|
||||
|
||||
|
||||
== Renamed Parameters
|
||||
|
||||
The community has decided to allow operators to specify units for Cassandra parameters of types duration, data storage, and data rate.
|
||||
All parameters which had a particular unit (most of the time added as a suffix to their name) can be now set by using the format [value][unit]. The unit suffix has been removed from their names.
|
||||
|
||||
*Example*:
|
||||
|
||||
Old name and value format:
|
||||
....
|
||||
permissions_update_interval_ms: 0
|
||||
....
|
||||
New name and possible value formats:
|
||||
....
|
||||
permissions_update_interval: 0ms
|
||||
permissions_update_interval: 0s
|
||||
permissions_update_interval: 0d
|
||||
permissions_update_interval: 0us
|
||||
permissions_update_interval: 0µs
|
||||
....
|
||||
|
||||
The work in https://issues.apache.org/jira/browse/CASSANDRA-15234[CASSANDRA-15234] was already quite big, so we decided
|
||||
to introduce the notion of the smallest allowed unit per parameter for duration and data storage parameters. What does this mean?
|
||||
Cassandra's internals still use the old units for parameters. If, for example, seconds are used internally, but you want
|
||||
to add a value in nanoseconds in `cassandra.yaml`, you will get a configuration exception that contains the following information:
|
||||
....
|
||||
Accepted units: seconds, minutes, hours, days.
|
||||
....
|
||||
|
||||
Why was this needed?
|
||||
Because we can run into precision issues. The full solution to the problem is to convert internally all parameters’ values
|
||||
to be manipulated with the smallest supported by Cassandra unit. A series of tickets to assess and migrate to the smallest unit
|
||||
our parameters (incrementally, post https://issues.apache.org/jira/browse/CASSANDRA-15234[CASSANDRA-15234]) will be opened soon.
|
||||
|
||||
|
||||
[cols=",,",options="header",]
|
||||
|===
|
||||
|Old Name |New Name |The Smallest Supported Unit
|
||||
|permissions_validity_in_ms |permissions_validity |ms
|
||||
|permissions_update_interval_in_ms |permissions_update_interval |ms
|
||||
|roles_validity_in_ms |roles_validity |ms
|
||||
|roles_update_interval_in_ms |roles_update_interval |ms
|
||||
|roles_update_interval_in_ms |roles_update_interval |ms
|
||||
|credentials_validity_in_ms |credentials_validity |ms
|
||||
|credentials_update_interval_in_ms |credentials_update_interval |ms
|
||||
|max_hint_window_in_ms |max_hint_window |ms
|
||||
|native_transport_idle_timeout_in_ms |native_transport_idle_timeout |ms
|
||||
|request_timeout_in_ms |request_timeout |ms
|
||||
|read_request_timeout_in_ms |read_request_timeout |ms
|
||||
|range_request_timeout_in_ms |range_request_timeout |ms
|
||||
|write_request_timeout_in_ms |write_request_timeout |ms
|
||||
|counter_write_request_timeout_in_ms |counter_write_request_timeout |ms
|
||||
|cas_contention_timeout_in_ms |cas_contention_timeout |ms
|
||||
|truncate_request_timeout_in_ms |truncate_request_timeout |ms
|
||||
|streaming_keep_alive_period_in_secs |streaming_keep_alive_period |s
|
||||
|cross_node_timeout |internode_timeout |-
|
||||
|slow_query_log_timeout_in_ms |slow_query_log_timeout |ms
|
||||
|memtable_heap_space_in_mb |memtable_heap_space |MiB
|
||||
|memtable_offheap_space_in_mb |memtable_offheap_space |MiB
|
||||
|repair_session_space_in_mb |repair_session_space |MiB
|
||||
|internode_max_message_size_in_bytes |internode_max_message_size |B
|
||||
|internode_send_buff_size_in_bytes |internode_socket_send_buffer_size |B
|
||||
|internode_socket_send_buffer_size_in_bytes |internode_socket_send_buffer_size |B
|
||||
|internode_socket_receive_buffer_size_in_bytes |internode_socket_receive_buffer_size |B
|
||||
|internode_recv_buff_size_in_bytes |internode_socket_receive_buffer_size |B
|
||||
|internode_application_send_queue_capacity_in_bytes |internode_application_send_queue_capacity |B
|
||||
|internode_application_send_queue_reserve_endpoint_capacity_in_bytes |internode_application_send_queue_reserve_endpoint_capacity |B
|
||||
|internode_application_send_queue_reserve_global_capacity_in_bytes |internode_application_send_queue_reserve_global_capacity |B
|
||||
|internode_application_receive_queue_capacity_in_bytes |internode_application_receive_queue_capacity |B
|
||||
|internode_application_receive_queue_reserve_endpoint_capacity_in_bytes |internode_application_receive_queue_reserve_endpoint_capacity |B
|
||||
|internode_application_receive_queue_reserve_global_capacity_in_bytes |internode_application_receive_queue_reserve_global_capacity |B
|
||||
|internode_tcp_connect_timeout_in_ms |internode_tcp_connect_timeout |ms
|
||||
|internode_tcp_user_timeout_in_ms |internode_tcp_user_timeout |ms
|
||||
|internode_streaming_tcp_user_timeout_in_ms |internode_streaming_tcp_user_timeout |ms
|
||||
|native_transport_max_frame_size_in_mb |native_transport_max_frame_size |MiB
|
||||
|max_value_size_in_mb |max_value_size |MiB
|
||||
|column_index_size_in_kb |column_index_size |KiB
|
||||
|column_index_cache_size_in_kb |column_index_cache_size |KiB
|
||||
|batch_size_warn_threshold_in_kb |batch_size_warn_threshold |KiB
|
||||
|batch_size_fail_threshold_in_kb |batch_size_fail_threshold |KiB
|
||||
|compaction_throughput_mb_per_sec |compaction_throughput |MiB/s
|
||||
|compaction_large_partition_warning_threshold_mb |compaction_large_partition_warning_threshold |MiB
|
||||
|min_free_space_per_drive_in_mb |min_free_space_per_drive |MiB
|
||||
|stream_throughput_outbound_megabits_per_sec |stream_throughput_outbound |MiB/s
|
||||
|inter_dc_stream_throughput_outbound_megabits_per_sec |inter_dc_stream_throughput_outbound |MiB/s
|
||||
|commitlog_total_space_in_mb |commitlog_total_space |MiB
|
||||
|commitlog_sync_group_window_in_ms |commitlog_sync_group_window |ms
|
||||
|commitlog_sync_period_in_ms |commitlog_sync_period |ms
|
||||
|commitlog_segment_size_in_mb |commitlog_segment_size |MiB
|
||||
|periodic_commitlog_sync_lag_block_in_ms |periodic_commitlog_sync_lag_block |ms
|
||||
|max_mutation_size_in_kb |max_mutation_size |KiB
|
||||
|cdc_total_space_in_mb |cdc_total_space |MiB
|
||||
|cdc_free_space_check_interval_ms |cdc_free_space_check_interval |ms
|
||||
|dynamic_snitch_update_interval_in_ms |dynamic_snitch_update_interval |ms
|
||||
|dynamic_snitch_reset_interval_in_ms |dynamic_snitch_reset_interval |ms
|
||||
|hinted_handoff_throttle_in_kb |hinted_handoff_throttle |KiB
|
||||
|batchlog_replay_throttle_in_kb |batchlog_replay_throttle |KiB
|
||||
|hints_flush_period_in_ms |hints_flush_period |ms
|
||||
|max_hints_file_size_in_mb |max_hints_file_size |MiB
|
||||
|trickle_fsync_interval_in_kb |trickle_fsync_interval |KiB
|
||||
|sstable_preemptive_open_interval_in_mb |sstable_preemptive_open_interval |MiB
|
||||
|key_cache_size_in_mb |key_cache_size |MiB
|
||||
|row_cache_size_in_mb |row_cache_size |MiB
|
||||
|counter_cache_size_in_mb |counter_cache_size |MiB
|
||||
|networking_cache_size_in_mb |networking_cache_size |MiB
|
||||
|file_cache_size_in_mb |file_cache_size |MiB
|
||||
|index_summary_capacity_in_mb |index_summary_capacity |MiB
|
||||
|index_summary_resize_interval_in_minutes |index_summary_resize_interval |m
|
||||
|gc_log_threshold_in_ms |gc_log_threshold |ms
|
||||
|gc_warn_threshold_in_ms |gc_warn_threshold |ms
|
||||
|tracetype_query_ttl |trace_type_query_ttl |s
|
||||
|tracetype_repair_ttl |trace_type_repair_ttl |s
|
||||
|prepared_statements_cache_size_mb |prepared_statements_cache_size |MiB
|
||||
|enable_user_defined_functions |user_defined_functions_enabled |-
|
||||
|enable_scripted_user_defined_functions |scripted_user_defined_functions_enabled |-
|
||||
|enable_materialized_views |materialized_views_enabled |-
|
||||
|enable_transient_replication |transient_replication_enabled |-
|
||||
|enable_sasi_indexes |sasi_indexes_enabled |-
|
||||
|enable_drop_compact_storage |drop_compact_storage_enabled |-
|
||||
|enable_user_defined_functions_threads |user_defined_functions_threads_enabled |-
|
||||
|enable_legacy_ssl_storage_port |legacy_ssl_storage_port_enabled |-
|
||||
|===
|
||||
|
||||
Another TO DO is to add JMX methods supporting the new format. However, we may abandon this if virtual tables support for
|
||||
configuration changes in the near future.
|
||||
|
||||
*Notes for Cassandra Developers*:
|
||||
|
||||
- Most of our parameters are already moved to the new framework as part of https://issues.apache.org/jira/browse/CASSANDRA-15234[CASSANDRA-15234].
|
||||
`@Replaces` is the annotation to be used when you make changes to any configuration parameters in `Config` class and `cassandra.yaml`, and you want to add backward
|
||||
compatibility with previous Cassandra versions. `Converters` class enumerates the different methods used for backward compatibility.
|
||||
`IDENTITY` is the one used for name change only. For more information about the other Converters, please, check the JavaDoc in the class.
|
||||
For backward compatibility virtual table `Settings` contains both the old and the new
|
||||
parameters with the old and the new value format. Only exception at the moment are the following three parameters: `key_cache_save_period`,
|
||||
`row_cache_save_period` and `counter_cache_save_period` which appear only once with the new value format.
|
||||
The old names and value format still can be used at least until the next major release. Deprecation warning is emitted on startup.
|
||||
If the parameter is of type duration, data rate or data storage, its value should be accompanied by a unit when new name is used.
|
||||
|
||||
- Please follow the new format `noun_verb` when adding new configuration parameters.
|
||||
|
||||
- Please consider adding any new parameters with the lowest supported by Cassandra unit.
|
||||
|
||||
- If for some reason you consider the smallest unit shouldn’t be the one that is supported as such in Cassandra, please,
|
||||
use the extended classes `SmallestDuration*`, `SmallestDataStorage*`.
|
||||
|
||||
- New parameters should be added as non-negative numbers.
|
||||
|
||||
- Any time you add @Replaces with a name change, we need to add an entry in this https://github.com/riptano/ccm/blob/808b6ca13526785b0fddfe1ead2383c060c4b8b6/ccmlib/common.py#L62[Python dictionary in CCM] to support the same backward compatibility as SnakeYAML.
|
||||
|
||||
Please follow the instructions in requirements.txt in the DTest repo how to retag CCM after committing any changes.
|
||||
You might want to test also with tagging in your repo to ensure that there will be no surprise after retagging the official CCM.
|
||||
Please be sure to run a full CI after any changes as CCM affects a few of our testing suites.
|
||||
|
||||
- Some configuration parameters are not announced in cassandra.yaml, but they are presented in the Config class for advanced users.
|
||||
Those also should be using the new framework and naming conventions.
|
||||
|
||||
- As we have backward compatibility, we didn’t have to rework all python DTests to set config in the new format, and we exercise
|
||||
the backward compatibility while testing. Please consider adding any new tests using the new names and value format though.
|
||||
|
||||
- In-JVM upgrade tests do not support per-version configuration at the moment, so we have to keep the old names and value format.
|
||||
Currently, if we try to use the new config for a newer version, that will be silently ignored and default config will be used.
|
||||
|
||||
- SnakeYAML supports overloading of parameters. This means that if you add a configuration parameter more than once in your `cassandra.yaml` -
|
||||
the latest occasion will be the one to load in Config during Cassandra startup. In order to make upgrades as less disruptive as possible,
|
||||
we continue supporting that behavior also with adding old and new names of a parameter into `cassandra.yaml`.
|
||||
|
||||
*Example*:
|
||||
|
||||
If you add the following to `cassandra.yaml`:
|
||||
....
|
||||
hinted_handoff_enabled: true
|
||||
enabled_hinted_handolff: false
|
||||
....
|
||||
|
||||
you will get loaded in `Config`:
|
||||
....
|
||||
hinted_handoff_enabled: false
|
||||
....
|
||||
|
||||
https://issues.apache.org/jira/browse/CASSANDRA-17379[CASSANDRA-17379] was opened to improve the user experience and deprecate the overloading.
|
||||
|
|
@ -85,9 +85,9 @@ arrival from other nodes in the cluster. These limits are applied to the
|
|||
on-wire size of the message being sent or received.
|
||||
. The basic per-link limit is consumed in isolation before any endpoint
|
||||
or global limit is imposed. Each node-pair has three links: urgent,
|
||||
small and large. So any given node may have a maximum of
|
||||
`N*3 * (internode_application_send_queue_capacity_in_bytes + internode_application_receive_queue_capacity_in_bytes)`
|
||||
messages queued without any coordination between them although in
|
||||
small and large. Any given node may have a maximum of
|
||||
`N*3 * (internode_application_send_queue_capacity in bytes + internode_application_receive_queue_capacity in bytes)`
|
||||
of messages data queued without any coordination between them although in
|
||||
practice, with token-aware routing, only RF*tokens nodes should need to
|
||||
communicate with significant bandwidth.
|
||||
. The per-endpoint limit is imposed on all messages exceeding the
|
||||
|
|
@ -99,12 +99,12 @@ following configuration settings have been added to `cassandra.yaml` for
|
|||
resource limits on queued messages.
|
||||
|
||||
....
|
||||
internode_application_send_queue_capacity_in_bytes: 4194304 #4MiB
|
||||
internode_application_send_queue_reserve_endpoint_capacity_in_bytes: 134217728 #128MiB
|
||||
internode_application_send_queue_reserve_global_capacity_in_bytes: 536870912 #512MiB
|
||||
internode_application_receive_queue_capacity_in_bytes: 4194304 #4MiB
|
||||
internode_application_receive_queue_reserve_endpoint_capacity_in_bytes: 134217728 #128MiB
|
||||
internode_application_receive_queue_reserve_global_capacity_in_bytes: 536870912 #512MiB
|
||||
internode_application_send_queue_capacity: 4MiB
|
||||
internode_application_send_queue_reserve_endpoint_capacity: 128MiB
|
||||
internode_application_send_queue_reserve_global_capacity: 512MiB
|
||||
internode_application_receive_queue_capacity: 4MiB
|
||||
internode_application_receive_queue_reserve_endpoint_capacity: 128MiB
|
||||
internode_application_receive_queue_reserve_global_capacity: 512MiB
|
||||
....
|
||||
|
||||
== Virtual Tables for Messaging Metrics
|
||||
|
|
|
|||
|
|
@ -83,7 +83,7 @@ eligibility for Zero Copy streaming.
|
|||
When enabled, it permits Cassandra to zero-copy stream entire eligible
|
||||
SSTables between nodes, including every component. This speeds up the
|
||||
network transfer significantly subject to throttling specified by
|
||||
`stream_throughput_outbound_megabits_per_sec`.
|
||||
`stream_throughput_outbound`.
|
||||
|
||||
Enabling this will reduce the GC pressure on sending and receiving node.
|
||||
While this feature tries to keep the disks balanced, it cannot guarantee
|
||||
|
|
@ -93,28 +93,28 @@ is enabled. Currently this can be used with Leveled Compaction.
|
|||
=== Configuring for Zero Copy Streaming
|
||||
|
||||
Throttling would reduce the streaming speed. The
|
||||
`stream_throughput_outbound_megabits_per_sec` throttles all outbound
|
||||
`stream_throughput_outbound` throttles all outbound
|
||||
streaming file transfers on a node to the given total throughput in
|
||||
Mbps. When unset, the default is 200 Mbps or 25 MB/s.
|
||||
Mbps. When unset, the default is 200 Mbps or 24 MiB/s.
|
||||
|
||||
....
|
||||
stream_throughput_outbound_megabits_per_sec: 200
|
||||
stream_throughput_outbound: 24MiB/s
|
||||
....
|
||||
|
||||
To run any Zero Copy streaming benchmark the
|
||||
`stream_throughput_outbound_megabits_per_sec` must be set to a really
|
||||
`stream_throughput_outbound` must be set to a really
|
||||
high value otherwise, throttling will be significant and the benchmark
|
||||
results will not be meaningful.
|
||||
|
||||
The `inter_dc_stream_throughput_outbound_megabits_per_sec` throttles all
|
||||
The `inter_dc_stream_throughput_outbound` throttles all
|
||||
streaming file transfer between the datacenters, this setting allows
|
||||
users to throttle inter dc stream throughput in addition to throttling
|
||||
all network stream traffic as configured with
|
||||
`stream_throughput_outbound_megabits_per_sec`. When unset, the default
|
||||
`stream_throughput_outbound`. When unset, the default
|
||||
is 200 Mbps or 25 MB/s.
|
||||
|
||||
....
|
||||
inter_dc_stream_throughput_outbound_megabits_per_sec: 200
|
||||
inter_dc_stream_throughput_outbound: 24MiB/s
|
||||
....
|
||||
|
||||
=== SSTable Components Streamed with Zero Copy Streaming
|
||||
|
|
|
|||
|
|
@ -47,7 +47,7 @@ must be enabled on each node in a cluster separately by setting the
|
|||
following configuration property in `cassandra.yaml`.
|
||||
|
||||
....
|
||||
enable_transient_replication: true
|
||||
transient_replication_enabled: true
|
||||
....
|
||||
|
||||
Transient replication may be configured with both `SimpleStrategy` and
|
||||
|
|
|
|||
|
|
@ -277,7 +277,7 @@ name | value
|
|||
auto_snapshot | true
|
||||
automatic_sstable_upgrade | false
|
||||
cluster_name | Test Cluster
|
||||
enable_transient_replication | false
|
||||
transient_replication_enabled | false
|
||||
hinted_handoff_enabled | true
|
||||
hints_directory | /home/ec2-user/cassandra/data/hints
|
||||
incremental_backups | false
|
||||
|
|
|
|||
|
|
@ -52,10 +52,10 @@ The following cassandra.yaml options are available for CDC:
|
|||
`cdc_raw_directory` (default: `$CASSANDRA_HOME/data/cdc_raw`)::
|
||||
Destination for CommitLogSegments to be moved after all corresponding
|
||||
memtables are flushed.
|
||||
`cdc_free_space_in_mb`: (default: min of 4096 and 1/8th volume space)::
|
||||
`cdc_total_space`: (default: min of 4096MiB and 1/8th volume space)::
|
||||
Calculated as sum of all active CommitLogSegments that permit CDC +
|
||||
all flushed CDC segments in `cdc_raw_directory`.
|
||||
`cdc_free_space_check_interval_ms` (default: 250)::
|
||||
`cdc_free_space_check_interval` (default: 250ms)::
|
||||
When at capacity, we limit the frequency with which we re-calculate
|
||||
the space taken up by `cdc_raw_directory` to prevent burning CPU
|
||||
cycles unnecessarily. Default is to check 4 times per second.
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ replicas acknowledge the mutation the coordinator responds successfully
|
|||
to the client. If a replica node is unavailable, however, the
|
||||
coordinator stores a hint locally to the filesystem for later
|
||||
application. New hints will be retained for up to
|
||||
`max_hint_window_in_ms` of downtime (defaults to `3 hours`). If the
|
||||
`max_hint_windowin_ms` of downtime (defaults to `3 h`). If the
|
||||
unavailable replica does return to the cluster before the window
|
||||
expires, the coordinator applies any pending hinted mutations against
|
||||
the replica to ensure that eventual consistency is maintained.
|
||||
|
|
@ -87,18 +87,18 @@ application is idempotent and cannot overwrite a future mutation.
|
|||
=== Hints for Timed Out Write Requests
|
||||
|
||||
Hints are also stored for write requests that time out. The
|
||||
`write_request_timeout_in_ms` setting in `cassandra.yaml` configures the
|
||||
`write_request_timeout` setting in `cassandra.yaml` configures the
|
||||
timeout for write requests.
|
||||
|
||||
[source,none]
|
||||
----
|
||||
write_request_timeout_in_ms: 2000
|
||||
write_request_timeout: 2000ms
|
||||
----
|
||||
|
||||
The coordinator waits for the configured amount of time for write
|
||||
requests to complete, at which point it will time out and generate a
|
||||
hint for the timed out request. The lowest acceptable value for
|
||||
`write_request_timeout_in_ms` is 10 ms.
|
||||
`write_request_timeout` is 10 ms.
|
||||
|
||||
== Configuring Hints
|
||||
|
||||
|
|
@ -127,15 +127,15 @@ hinted_handoff_disabled_datacenters:
|
|||
|
||||
|`unset`
|
||||
|
||||
|`max_hint_window_in_ms` |Defines the maximum amount of time (ms) a node
|
||||
shall have hints generated after it has failed. |`10800000` # 3 hours
|
||||
|`max_hint_window` |Defines the maximum amount of time a node
|
||||
shall have hints generated after it has failed. |`3h`
|
||||
|
||||
|`hinted_handoff_throttle_in_kb` |Maximum throttle in KBs per second,
|
||||
|`hinted_handoff_throttle` |Maximum throttle in KiBs per second,
|
||||
per delivery thread. This will be reduced proportionally to the number
|
||||
of nodes in the cluster. (If there are two nodes in the cluster, each
|
||||
delivery thread will use the maximum rate; if there are 3, each will
|
||||
throttle to half of the maximum,since it is expected for two nodes to be
|
||||
delivering hints simultaneously.) |`1024`
|
||||
delivering hints simultaneously.) |`1024KiB`
|
||||
|
||||
|`max_hints_delivery_threads` |Number of threads with which to deliver
|
||||
hints; Consider increasing this number when you have multi-dc
|
||||
|
|
@ -144,11 +144,11 @@ deployments, since cross-dc handoff tends to be slower |`2`
|
|||
|`hints_directory` |Directory where Cassandra stores hints.
|
||||
|`$CASSANDRA_HOME/data/hints`
|
||||
|
||||
|`hints_flush_period_in_ms` |How often hints should be flushed from the
|
||||
internal buffers to disk. Will _not_ trigger fsync. |`10000`
|
||||
|`hints_flush_period` |How often hints should be flushed from the
|
||||
internal buffers to disk. Will _not_ trigger fsync. |`10000ms`
|
||||
|
||||
|`max_hints_file_size_in_mb` |Maximum size for a single hints file, in
|
||||
megabytes. |`128`
|
||||
|`max_hints_file_size |Maximum size for a single hints file, in
|
||||
megabytes. |`128MiB`
|
||||
|
||||
|`hints_compression` |Compression to apply to the hint files. If
|
||||
omitted, hints files will be written uncompressed. LZ4, Snappy, and
|
||||
|
|
@ -222,9 +222,9 @@ If you find yourself in such a situation, you may consider raising the
|
|||
=== Allow a Node to be Down Longer at Runtime
|
||||
|
||||
Sometimes a node may be down for more than the normal
|
||||
`max_hint_window_in_ms`, (default of three hours), but the hardware and
|
||||
`max_hint_window`, (default of three hours), but the hardware and
|
||||
data itself will still be accessible. In such a case you may consider
|
||||
raising the `max_hint_window_in_ms` dynamically via the
|
||||
raising the `max_hint_window` dynamically via the
|
||||
`nodetool setmaxhintwindow` command added in Cassandra 4.0
|
||||
(https://issues.apache.org/jira/browse/CASSANDRA-11720[CASSANDRA-11720]).
|
||||
This will instruct Cassandra to continue holding hints for the down
|
||||
|
|
@ -232,7 +232,7 @@ endpoint for a longer amount of time.
|
|||
|
||||
This command should be applied on all nodes in the cluster that may be
|
||||
holding hints. If needed, the setting can be applied permanently by
|
||||
setting the `max_hint_window_in_ms` setting in `cassandra.yaml` followed
|
||||
setting the `max_hint_window` setting in `cassandra.yaml` followed
|
||||
by a rolling restart.
|
||||
|
||||
== Monitoring Hint Delivery
|
||||
|
|
|
|||
|
|
@ -111,10 +111,10 @@ period from when the node initially dies to when a new node completes
|
|||
the replacement process.
|
||||
|
||||
[arabic]
|
||||
. The node is down for longer than `max_hint_window_in_ms` before being
|
||||
. The node is down for longer than `max_hint_window` before being
|
||||
replaced.
|
||||
. You are replacing using the same IP address as the dead node *and*
|
||||
replacement takes longer than `max_hint_window_in_ms`.
|
||||
replacement takes longer than `max_hint_window`.
|
||||
====
|
||||
|
||||
== Monitoring progress
|
||||
|
|
|
|||
|
|
@ -89,7 +89,7 @@ streaming throughput and client/server SSL
|
|||
|
||||
You can provide a cassandra.yaml file with the -f command line option to
|
||||
set up streaming throughput, and client and server encryption options.
|
||||
Only stream_throughput_outbound_megabits_per_sec,
|
||||
Only stream_throughput_outbound,
|
||||
server_encryption_options, and client_encryption_options are read from
|
||||
yaml. You can override options read from cassandra.yaml with
|
||||
corresponding command line options.
|
||||
|
|
|
|||
Loading…
Reference in New Issue