diff --git a/doc/source/operating/metrics.rst b/doc/source/operating/metrics.rst index 83661a2557..e3af955857 100644 --- a/doc/source/operating/metrics.rst +++ b/doc/source/operating/metrics.rst @@ -72,7 +72,7 @@ Reported name format: ``org.apache.cassandra.metrics.Table...`` **JMX MBean** - ``org.apache.cassandra.metrics:type=Table keyspace= scope=
name=`` + ``org.apache.cassandra.metrics:type=Table,keyspace=,scope=
,name=`` .. NOTE:: There is a special table called '``all``' without a keyspace. This represents the aggregation of metrics across @@ -82,83 +82,92 @@ Reported name format: =============================================== ============== =========== Name Type Description =============================================== ============== =========== -MemtableOnHeapSize Gauge Total amount of data stored in the memtable that resides **on**-heap, including column related overhead and partitions overwritten. -MemtableOffHeapSize Gauge Total amount of data stored in the memtable that resides **off**-heap, including column related overhead and partitions overwritten. -MemtableLiveDataSize Gauge Total amount of live data stored in the memtable, excluding any data structure overhead. -AllMemtablesOnHeapSize Gauge Total amount of data stored in the memtables (2i and pending flush memtables included) that resides **on**-heap. -AllMemtablesOffHeapSize Gauge Total amount of data stored in the memtables (2i and pending flush memtables included) that resides **off**-heap. +AdditionalWrites Counter Total number of additional writes sent to the replicas (other than the intial contacted ones). AllMemtablesLiveDataSize Gauge Total amount of live data stored in the memtables (2i and pending flush memtables included) that resides off-heap, excluding any data structure overhead. -MemtableColumnsCount Gauge Total number of columns present in the memtable. -MemtableSwitchCount Counter Number of times flush has resulted in the memtable being switched out. -CompressionRatio Gauge Current compression ratio for all SSTables. -EstimatedPartitionSizeHistogram Gauge Histogram of estimated partition size (in bytes). -EstimatedPartitionCount Gauge Approximate number of keys in table. -EstimatedColumnCountHistogram Gauge Histogram of estimated number of columns. -SSTablesPerReadHistogram Histogram Histogram of the number of sstable data files accessed per single partition read. SSTables skipped due to Bloom Filters, min-max key or partition index lookup are not taken into acoount. -ReadLatency Latency Local read latency for this table. -RangeLatency Latency Local range scan latency for this table. -WriteLatency Latency Local write latency for this table. -CoordinatorReadLatency Timer Coordinator read latency for this table. -CoordinatorWriteLatency Timer Coordinator write latency for this table. -CoordinatorScanLatency Timer Coordinator range scan latency for this table. -PendingFlushes Counter Estimated number of flush tasks pending for this table. -BytesFlushed Counter Total number of bytes flushed since server [re]start. -CompactionBytesWritten Counter Total number of bytes written by compaction since server [re]start. -PendingCompactions Gauge Estimate of number of pending compactions for this table. -LiveSSTableCount Gauge Number of SSTables on disk for this table. -LiveDiskSpaceUsed Counter Disk space used by SSTables belonging to this table (in bytes). -TotalDiskSpaceUsed Counter Total disk space used by SSTables belonging to this table, including obsolete ones waiting to be GC'd. -MinPartitionSize Gauge Size of the smallest compacted partition (in bytes). -MaxPartitionSize Gauge Size of the largest compacted partition (in bytes). -MeanPartitionSize Gauge Size of the average compacted partition (in bytes). +AllMemtablesOffHeapDataSize Gauge Total amount of data stored in the memtables (2i and pending flush memtables included) that resides **off**-heap. +AllMemtablesOffHeapSize (Deprecated) Gauge Total amount of data stored in the memtables (2i and pending flush memtables included) that resides **off**-heap. +AllMemtablesOnHeapDataSize Gauge Total amount of data stored in the memtables (2i and pending flush memtables included) that resides **on**-heap. +AllMemtablesOnHeapSize (Deprecated) Gauge Total amount of data stored in the memtables (2i and pending flush memtables included) that resides **on**-heap. +AnticompactionTime Timer Time spent anticompacting before a consistent repair. +BloomFilterDiskSpaceUsed Gauge Disk space used by bloom filter (in bytes). BloomFilterFalsePositives Gauge Number of false positives on table's bloom filter. BloomFilterFalseRatio Gauge False positive ratio of table's bloom filter. -BloomFilterDiskSpaceUsed Gauge Disk space used by bloom filter (in bytes). BloomFilterOffHeapMemoryUsed Gauge Off-heap memory used by bloom filter. -IndexSummaryOffHeapMemoryUsed Gauge Off-heap memory used by index summary. -CompressionMetadataOffHeapMemoryUsed Gauge Off-heap memory used by compression meta data. -KeyCacheHitRate Gauge Key cache hit rate for this table. -TombstoneScannedHistogram Histogram Histogram of tombstones scanned in queries on this table. -LiveScannedHistogram Histogram Histogram of live cells scanned in queries on this table. -ColUpdateTimeDeltaHistogram Histogram Histogram of column update time delta on this table. -ViewLockAcquireTime Timer Time taken acquiring a partition lock for materialized view updates on this table. -ViewReadTime Timer Time taken during the local read of a materialized view update. -TrueSnapshotsSize Gauge Disk space used by snapshots of this table including all SSTable components. -RowCacheHitOutOfRange Counter Number of table row cache hits that do not satisfy the query filter, thus went to disk. -RowCacheHit Counter Number of table row cache hits. -RowCacheMiss Counter Number of table row cache misses. -CasPrepare Latency Latency of paxos prepare round. -CasPropose Latency Latency of paxos propose round. -CasCommit Latency Latency of paxos commit round. -PercentRepaired Gauge Percent of table data that is repaired on disk. +BytesAnticompacted Counter How many bytes we anticompacted. +BytesFlushed Counter Total number of bytes flushed since server [re]start. +BytesMutatedAnticompaction Counter How many bytes we avoided anticompacting because the sstable was fully contained in the repaired range. +BytesPendingRepair Gauge Size of table data isolated for an ongoing incremental repair BytesRepaired Gauge Size of table data repaired on disk BytesUnrepaired Gauge Size of table data unrepaired on disk -BytesPendingRepair Gauge Size of table data isolated for an ongoing incremental repair -SpeculativeRetries Counter Number of times speculative retries were sent for this table. -SpeculativeFailedRetries Counter Number of speculative retries that failed to prevent a timeout -SpeculativeInsufficientReplicas Counter Number of speculative retries that couldn't be attempted due to lack of replicas -SpeculativeSampleLatencyNanos Gauge Number of nanoseconds to wait before speculation is attempted. Value may be statically configured or updated periodically based on coordinator latency. -WaitingOnFreeMemtableSpace Histogram Histogram of time spent waiting for free memtable space, either on- or off-heap. -DroppedMutations Counter Number of dropped mutations on this table. -AnticompactionTime Timer Time spent anticompacting before a consistent repair. -ValidationTime Timer Time spent doing validation compaction during repair. -SyncTime Timer Time spent doing streaming during repair. BytesValidated Histogram Histogram over the amount of bytes read during validation. -PartitionsValidated Histogram Histogram over the number of partitions read during validation. -BytesAnticompacted Counter How many bytes we anticompacted. -BytesMutatedAnticompaction Counter How many bytes we avoided anticompacting because the sstable was fully contained in the repaired range. +CasCommit Latency Latency of paxos commit round. +CasPrepare Latency Latency of paxos prepare round. +CasPropose Latency Latency of paxos propose round. +ColUpdateTimeDeltaHistogram Histogram Histogram of column update time delta on this table. +CompactionBytesWritten Counter Total number of bytes written by compaction since server [re]start. +CompressionMetadataOffHeapMemoryUsed Gauge Off-heap memory used by compression meta data. +CompressionRatio Gauge Current compression ratio for all SSTables. +CoordinatorReadLatency Timer Coordinator read latency for this table. +CoordinatorScanLatency Timer Coordinator range scan latency for this table. +CoordinatorWriteLatency Timer Coordinator write latency for this table. +DroppedMutations Counter Number of dropped mutations on this table. +EstimatedColumnCountHistogram Gauge Histogram of estimated number of columns. +EstimatedPartitionCount Gauge Approximate number of keys in table. +EstimatedPartitionSizeHistogram Gauge Histogram of estimated partition size (in bytes). +IndexSummaryOffHeapMemoryUsed Gauge Off-heap memory used by index summary. +KeyCacheHitRate Gauge Key cache hit rate for this table. +LiveDiskSpaceUsed Counter Disk space used by SSTables belonging to this table (in bytes). +LiveSSTableCount Gauge Number of SSTables on disk for this table. +LiveScannedHistogram Histogram Histogram of live cells scanned in queries on this table. +MaxPartitionSize Gauge Size of the largest compacted partition (in bytes). +MeanPartitionSize Gauge Size of the average compacted partition (in bytes). +MemtableColumnsCount Gauge Total number of columns present in the memtable. +MemtableLiveDataSize Gauge Total amount of live data stored in the memtable, excluding any data structure overhead. +MemtableOffHeapDataSize Gauge Total amount of data stored in the memtable that resides **off**-heap, including column related overhead and partitions overwritten. +MemtableOffHeapSize (Deprecated) Gauge Total amount of data stored in the memtable that resides **off**-heap, including column related overhead and partitions overwritten. +MemtableOnHeapDataSize Gauge Total amount of data stored in the memtable that resides **on**-heap, including column related overhead and partitions overwritten. +MemtableOnHeapSize (Deprecated) Gauge Total amount of data stored in the memtable that resides **on**-heap, including column related overhead and partitions overwritten. +MemtableSwitchCount Counter Number of times flush has resulted in the memtable being switched out. +MinPartitionSize Gauge Size of the smallest compacted partition (in bytes). MutatedAnticompactionGauge Gauge Ratio of bytes mutated vs total bytes repaired. +PartitionsValidated Histogram Histogram over the number of partitions read during validation. +PendingCompactions Gauge Estimate of number of pending compactions for this table. +PendingFlushes Counter Estimated number of flush tasks pending for this table. +PercentRepaired Gauge Percent of table data that is repaired on disk. +RangeLatency Latency Local range scan latency for this table. +ReadLatency Latency Local read latency for this table. ReadRepairRequests Meter Throughput for mutations generated by read-repair. -ShortReadProtectionRequests Meter Throughput for requests to get extra rows during short read protection. +RepairJobsCompleted Counter Total number of completed repairs as coordinator on this table. +RepairJobsStarted Counter Total number of started repairs as coordinator on this table. +RepairSyncTime Timer Time spent doing streaming during repair. +RepairedDataTrackingOverreadRows Histogram Histogram of the amount of the additonal rows of the repaired data read. +RepairedDataTrackingOverreadTime Timer Time spent reading the additional rows of the repaired data. ReplicaFilteringProtectionRequests Meter Throughput for row completion requests during replica filtering protection. ReplicaFilteringProtectionRowsCachedPerQuery Histogram Histogram of the number of rows cached per query when replica filtering protection is engaged. +RowCacheHit Counter Number of table row cache hits. +RowCacheHitOutOfRange Counter Number of table row cache hits that do not satisfy the query filter, thus went to disk. +RowCacheMiss Counter Number of table row cache misses. +SSTablesPerReadHistogram Histogram Histogram of the number of sstable data files accessed per single partition read. SSTables skipped due to Bloom Filters, min-max key or partition index lookup are not taken into acoount. +ShortReadProtectionRequests Meter Throughput for requests to get extra rows during short read protection. +SpeculativeFailedRetries Counter Number of speculative retries that failed to prevent a timeout +SpeculativeInsufficientReplicas Counter Number of speculative retries that couldn't be attempted due to lack of replicas +SpeculativeRetries Counter Number of times speculative retries were sent for this table. +SpeculativeSampleLatencyNanos Gauge Number of nanoseconds to wait before speculation is attempted. Value may be statically configured or updated periodically based on coordinator latency. +TombstoneScannedHistogram Histogram Histogram of tombstones scanned in queries on this table. +TotalDiskSpaceUsed Counter Total disk space used by SSTables belonging to this table, including obsolete ones waiting to be GC'd. +TrueSnapshotsSize Gauge Disk space used by snapshots of this table including all SSTable components. +ValidationTime Timer Time spent doing validation compaction during repair. +ViewLockAcquireTime Timer Time taken acquiring a partition lock for materialized view updates on this table. +ViewReadTime Timer Time taken during the local read of a materialized view update. +WaitingOnFreeMemtableSpace Histogram Histogram of time spent waiting for free memtable space, either on- or off-heap. +WriteLatency Latency Local write latency for this table. =============================================== ============== =========== Keyspace Metrics ^^^^^^^^^^^^^^^^ Each keyspace in Cassandra has metrics responsible for tracking its state and performance. -Most of these metrics are the same as the ``Table Metrics`` above, only they are aggregated at the Keyspace level. The keyspace specific metrics are specified in the table below. +Most of these metrics are the same as the ``Table Metrics`` above, only they are aggregated at the Keyspace level. Only the keyspace specific metrics are specified in the table below. Reported name format: @@ -166,16 +175,16 @@ Reported name format: ``org.apache.cassandra.metrics.keyspace..`` **JMX MBean** - ``org.apache.cassandra.metrics:type=Keyspace scope= name=`` + ``org.apache.cassandra.metrics:type=Keyspace,keyspace=,name=`` ======================================= ============== =========== Name Type Description ======================================= ============== =========== -WriteFailedIdeaCL Counter Number of writes that failed to achieve the configured ideal consistency level or 0 if none is configured IdealCLWriteLatency Latency Coordinator latency of writes at the configured ideal consistency level. No values are recorded if ideal consistency level is not configured -RepairTime Timer Total time spent as repair coordinator. RepairPrepareTime Timer Total time spent preparing for repair. +RepairTime Timer Total time spent as repair coordinator. +WriteFailedIdealCL Counter Number of writes that failed to achieve the configured ideal consistency level or 0 if none is configured ======================================= ============== =========== ThreadPool Metrics @@ -194,18 +203,18 @@ Reported name format: ``org.apache.cassandra.metrics.ThreadPools...`` **JMX MBean** - ``org.apache.cassandra.metrics:type=ThreadPools path= scope= name=`` + ``org.apache.cassandra.metrics:type=ThreadPools,path=,scope=,name=`` ===================== ============== =========== Name Type Description ===================== ============== =========== ActiveTasks Gauge Number of tasks being actively worked on by this pool. -PendingTasks Gauge Number of queued tasks queued up on this pool. CompletedTasks Counter Number of tasks completed. -TotalBlockedTasks Counter Number of tasks that were blocked due to queue saturation. CurrentlyBlockedTask Counter Number of tasks that are currently blocked due to queue saturation but on retry will become unblocked. MaxPoolSize Gauge The maximum number of threads in this pool. MaxTasksQueued Gauge The maximum number of tasks queued before a task get blocked. +PendingTasks Gauge Number of queued tasks queued up on this pool. +TotalBlockedTasks Counter Number of tasks that were blocked due to queue saturation. ===================== ============== =========== The following thread pools can be monitored. @@ -213,16 +222,10 @@ The following thread pools can be monitored. ============================ ============== =========== Name Type Description ============================ ============== =========== -Native-Transport-Requests transport Handles client CQL requests -CounterMutationStage request Responsible for counter writes -ViewMutationStage request Responsible for materialized view writes -MutationStage request Responsible for all other writes -ReadRepairStage request ReadRepair happens on this thread pool -ReadStage request Local reads run on this thread pool -RequestResponseStage request Coordinator requests to the cluster run on this thread pool AntiEntropyStage internal Builds merkle tree for repairs CacheCleanupExecutor internal Cache maintenance performed on this thread pool CompactionExecutor internal Compactions are run on these threads +CounterMutationStage request Responsible for counter writes GossipStage internal Handles gossip requests HintsDispatcher internal Performs hinted handoff InternalResponseStage internal Responsible for intra-cluster callbacks @@ -231,12 +234,18 @@ MemtablePostFlush internal Cleans up commit log after memtable MemtableReclaimMemory internal Memtable recycling MigrationStage internal Runs schema migrations MiscStage internal Misceleneous tasks run here +MutationStage request Responsible for all other writes +Native-Transport-Requests transport Handles client CQL requests PendingRangeCalculator internal Calculates token range PerDiskMemtableFlushWriter_0 internal Responsible for writing a spec (there is one of these per disk 0-N) +ReadRepairStage request ReadRepair happens on this thread pool +ReadStage request Local reads run on this thread pool +RequestResponseStage request Coordinator requests to the cluster run on this thread pool Sampler internal Responsible for re-sampling the index summaries of SStables SecondaryIndexManagement internal Performs updates to secondary indexes ValidationExecutor internal Performs validation compaction or scrubbing ViewBuildExecutor internal Performs materialized views initial build +ViewMutationStage request Responsible for materialized view writes ============================ ============== =========== .. |nbsp| unicode:: 0xA0 .. nonbreaking space @@ -254,7 +263,7 @@ Reported name format: ``org.apache.cassandra.metrics.ClientRequest..`` **JMX MBean** - ``org.apache.cassandra.metrics:type=ClientRequest scope= name=`` + ``org.apache.cassandra.metrics:type=ClientRequest,scope=,name=`` :RequestType: CASRead @@ -263,13 +272,13 @@ Reported name format: ===================== ============== ============================================================= Name Type Description ===================== ============== ============================================================= - Timeouts Counter Number of timeouts encountered. - Failures Counter Number of transaction failures encountered. - |nbsp| Latency Transaction read latency. - Unavailables Counter Number of unavailable exceptions encountered. - UnfinishedCommit Counter Number of transactions that were committed on read. ConditionNotMet Counter Number of transaction preconditions did not match current values. ContentionHistogram Histogram How many contended reads were encountered + Failures Counter Number of transaction failures encountered. + Timeouts Counter Number of timeouts encountered. + Unavailables Counter Number of unavailable exceptions encountered. + UnfinishedCommit Counter Number of transactions that were committed on read. + |nbsp| Latency Transaction read latency. ===================== ============== ============================================================= :RequestType: CASWrite @@ -278,13 +287,13 @@ Reported name format: ===================== ============== ============================================================= Name Type Description ===================== ============== ============================================================= - Timeouts Counter Number of timeouts encountered. - Failures Counter Number of transaction failures encountered. - |nbsp| Latency Transaction write latency. - UnfinishedCommit Counter Number of transactions that were committed on write. ConditionNotMet Counter Number of transaction preconditions did not match current values. ContentionHistogram Histogram How many contended writes were encountered + Failures Counter Number of transaction failures encountered. MutationSizeHistogram Histogram Total size in bytes of the requests mutations. + Timeouts Counter Number of timeouts encountered. + UnfinishedCommit Counter Number of transactions that were committed on write. + |nbsp| Latency Transaction write latency. ===================== ============== ============================================================= @@ -294,10 +303,10 @@ Reported name format: ===================== ============== ============================================================= Name Type Description ===================== ============== ============================================================= - Timeouts Counter Number of timeouts encountered. Failures Counter Number of read failures encountered. - |nbsp| Latency Read latency. + Timeouts Counter Number of timeouts encountered. Unavailables Counter Number of unavailable exceptions encountered. + |nbsp| Latency Read latency. ===================== ============== ============================================================= :RequestType: RangeSlice @@ -306,10 +315,10 @@ Reported name format: ===================== ============== ============================================================= Name Type Description ===================== ============== ============================================================= - Timeouts Counter Number of timeouts encountered. Failures Counter Number of range query failures encountered. - |nbsp| Latency Range query latency. + Timeouts Counter Number of timeouts encountered. Unavailables Counter Number of unavailable exceptions encountered. + |nbsp| Latency Range query latency. ===================== ============== ============================================================= :RequestType: Write @@ -318,11 +327,11 @@ Reported name format: ===================== ============== ============================================================= Name Type Description ===================== ============== ============================================================= - Timeouts Counter Number of timeouts encountered. Failures Counter Number of write failures encountered. - |nbsp| Latency Write latency. - Unavailables Counter Number of unavailable exceptions encountered. MutationSizeHistogram Histogram Total size in bytes of the requests mutations. + Timeouts Counter Number of timeouts encountered. + Unavailables Counter Number of unavailable exceptions encountered. + |nbsp| Latency Write latency. ===================== ============== ============================================================= @@ -330,12 +339,12 @@ Reported name format: :Description: Metrics related to materialized view write wrtes. :Metrics: ===================== ============== ============================================================= - Timeouts Counter Number of timeouts encountered. Failures Counter Number of transaction failures encountered. + Timeouts Counter Number of timeouts encountered. Unavailables Counter Number of unavailable exceptions encountered. + ViewPendingMutations Gauge ViewReplicasAttempted - ViewReplicasSuccess. ViewReplicasAttempted Counter Total number of attempted view replica writes. ViewReplicasSuccess Counter Total number of succeded view replica writes. - ViewPendingMutations Gauge ViewReplicasAttempted - ViewReplicasSuccess. ViewWriteLatency Timer Time between when mutation is applied to base table and when CL.ONE is achieved on view. ===================== ============== ============================================================= @@ -350,7 +359,7 @@ Reported name format: ``org.apache.cassandra.metrics.Cache..`` **JMX MBean** - ``org.apache.cassandra.metrics:type=Cache scope= name=`` + ``org.apache.cassandra.metrics:type=Cache,scope=,name=`` ========================== ============== =========== Name Type Description @@ -359,11 +368,11 @@ Capacity Gauge Cache capacity in bytes. Entries Gauge Total number of cache entries. FifteenMinuteCacheHitRate Gauge 15m cache hit rate. FiveMinuteCacheHitRate Gauge 5m cache hit rate. -OneMinuteCacheHitRate Gauge 1m cache hit rate. HitRate Gauge All time cache hit rate. Hits Meter Total number of cache hits. -Misses Meter Total number of cache misses. MissLatency Timer Latency of misses. +Misses Meter Total number of cache misses. +OneMinuteCacheHitRate Gauge 1m cache hit rate. Requests Gauge Total number of cache requests. Size Gauge Total size of occupied cache, in bytes. ========================== ============== =========== @@ -373,8 +382,8 @@ The following caches are covered: ============================ =========== Name Description ============================ =========== -CounterCache Keeps hot counters in memory for performance. ChunkCache In process uncompressed page cache. +CounterCache Keeps hot counters in memory for performance. KeyCache Cache for partition to sstable offsets. RowCache Cache for rows kept in memory. ============================ =========== @@ -393,7 +402,7 @@ Reported name format: ``org.apache.cassandra.metrics.CQL.`` **JMX MBean** - ``org.apache.cassandra.metrics:type=CQL name=`` + ``org.apache.cassandra.metrics:type=CQL,name=`` ========================== ============== =========== Name Type Description @@ -401,8 +410,8 @@ Name Type Description PreparedStatementsCount Gauge Number of cached prepared statements. PreparedStatementsEvicted Counter Number of prepared statements evicted from the prepared statement cache PreparedStatementsExecuted Counter Number of prepared statements executed. -RegularStatementsExecuted Counter Number of **non** prepared statements executed. PreparedStatementsRatio Gauge Percentage of statements that are prepared vs unprepared. +RegularStatementsExecuted Counter Number of **non** prepared statements executed. ========================== ============== =========== .. _dropped-metrics: @@ -419,14 +428,14 @@ Reported name format: ``org.apache.cassandra.metrics.DroppedMessage..`` **JMX MBean** - ``org.apache.cassandra.metrics:type=DroppedMessage scope= name=`` + ``org.apache.cassandra.metrics:type=DroppedMessage,scope=,name=`` ========================== ============== =========== Name Type Description ========================== ============== =========== CrossNodeDroppedLatency Timer The dropped latency across nodes. -InternalDroppedLatency Timer The dropped latency within node. Dropped Meter Number of dropped messages. +InternalDroppedLatency Timer The dropped latency within node. ========================== ============== =========== The different types of messages tracked are: @@ -434,15 +443,15 @@ The different types of messages tracked are: ============================ =========== Name Description ============================ =========== -BATCH_STORE Batchlog write BATCH_REMOVE Batchlog cleanup (after succesfully applied) +BATCH_STORE Batchlog write COUNTER_MUTATION Counter writes HINT Hint replay MUTATION Regular writes -READ Regular reads -READ_REPAIR Read repair PAGED_SLICE Paged read RANGE_SLICE Token range read +READ Regular reads +READ_REPAIR Read repair REQUEST_RESPONSE RPC Callbacks _TRACE Tracing writes ============================ =========== @@ -460,14 +469,14 @@ Reported name format: ``org.apache.cassandra.metrics.Streaming..`` **JMX MBean** - ``org.apache.cassandra.metrics:type=Streaming scope= name=`` + ``org.apache.cassandra.metrics:type=Streaming,scope=,name=`` ========================== ============== =========== Name Type Description ========================== ============== =========== IncomingBytes Counter Number of bytes streamed to this node from the peer. -OutgoingBytes Counter Number of bytes streamed to the peer endpoint from this node. IncomingProcessTime Timer The time spent on processing the incoming stream message from the peer. +OutgoingBytes Counter Number of bytes streamed to the peer endpoint from this node. ========================== ============== =========== @@ -482,16 +491,16 @@ Reported name format: ``org.apache.cassandra.metrics.Compaction.`` **JMX MBean** - ``org.apache.cassandra.metrics:type=Compaction name=`` + ``org.apache.cassandra.metrics:type=Compaction,name=`` ========================== ======================================== =============================================== Name Type Description ========================== ======================================== =============================================== BytesCompacted Counter Total number of bytes compacted since server [re]start. -PendingTasks Gauge Estimated number of compactions remaining to perform. CompletedTasks Gauge Number of completed compactions since server [re]start. -TotalCompactionsCompleted Meter Throughput of completed compactions since server [re]start. +PendingTasks Gauge Estimated number of compactions remaining to perform. PendingTasksByTableName Gauge>> Estimated number of compactions remaining to perform, grouped by keyspace and then table name. This info is also kept in ``Table Metrics``. +TotalCompactionsCompleted Meter Throughput of completed compactions since server [re]start. ========================== ======================================== =============================================== CommitLog Metrics @@ -505,17 +514,17 @@ Reported name format: ``org.apache.cassandra.metrics.CommitLog.`` **JMX MBean** - ``org.apache.cassandra.metrics:type=CommitLog name=`` + ``org.apache.cassandra.metrics:type=CommitLog,name=`` ========================== ============== =========== Name Type Description ========================== ============== =========== CompletedTasks Gauge Total number of commit log messages written since [re]start. +OverSizedMutations Meter Throughput for mutations that exceed limit. PendingTasks Gauge Number of commit log messages written but yet to be fsync'd. TotalCommitLogSize Gauge Current size, in bytes, used by all the commit log segments. +WaitingOnCommit Timer Time spent waiting on CL fsync; for Periodic this is only occurs when the sync is lagging its sync interval. WaitingOnSegmentAllocation Timer Time spent waiting for a CommitLogSegment to be allocated - under normal conditions this should be zero. -WaitingOnCommit Timer The time spent waiting on CL fsync; for Periodic this is only occurs when the sync is lagging its sync interval. -OverSizedMutations Meter Throughput for mutations that exceed limit. ========================== ============== =========== Storage Metrics @@ -529,7 +538,7 @@ Reported name format: ``org.apache.cassandra.metrics.Storage.`` **JMX MBean** - ``org.apache.cassandra.metrics:type=Storage name=`` + ``org.apache.cassandra.metrics:type=Storage,name=`` ========================== ============== =========== Name Type Description @@ -555,16 +564,16 @@ Reported name format: ``org.apache.cassandra.metrics.HintsService.`` **JMX MBean** - ``org.apache.cassandra.metrics:type=HintsService name=`` + ``org.apache.cassandra.metrics:type=HintsService,name=`` =========================== ============== =========== Name Type Description =========================== ============== =========== -HintsSucceeded Meter A meter of the hints successfully delivered -HintsFailed Meter A meter of the hints that failed deliver -HintsTimedOut Meter A meter of the hints that timed out Hint_delays Histogram Histogram of hint delivery delays (in milliseconds) Hint_delays- Histogram Histogram of hint delivery delays (in milliseconds) per peer +HintsFailed Meter A meter of the hints that failed deliver +HintsSucceeded Meter A meter of the hints successfully delivered +HintsTimedOut Meter A meter of the hints that timed out =========================== ============== =========== SSTable Index Metrics @@ -578,14 +587,14 @@ Reported name format: ``org.apache.cassandra.metrics.Index..RowIndexEntry`` **JMX MBean** - ``org.apache.cassandra.metrics:type=Index scope=RowIndexEntry name=`` + ``org.apache.cassandra.metrics:type=Index scope=RowIndexEntry,name=`` =========================== ============== =========== Name Type Description =========================== ============== =========== -IndexedEntrySize Histogram Histogram of the on-heap size, in bytes, of the index across all SSTables. IndexInfoCount Histogram Histogram of the number of on-heap index entries managed across all SSTables. IndexInfoGets Histogram Histogram of the number index seeks performed per SSTable. +IndexedEntrySize Histogram Histogram of the on-heap size, in bytes, of the index across all SSTables. =========================== ============== =========== BufferPool Metrics @@ -600,16 +609,15 @@ Reported name format: ``org.apache.cassandra.metrics.BufferPool.`` **JMX MBean** - ``org.apache.cassandra.metrics:type=BufferPool name=`` + ``org.apache.cassandra.metrics:type=BufferPool,name=`` =========================== ============== =========== Name Type Description =========================== ============== =========== +Misses Meter The rate of misses in the pool. The higher this is the more allocations incurred. Size Gauge Size, in bytes, of the managed buffer pool -Misses Meter The rate of misses in the pool. The higher this is the more allocations incurred. =========================== ============== =========== - Client Metrics ^^^^^^^^^^^^^^ @@ -621,24 +629,23 @@ Reported name format: ``org.apache.cassandra.metrics.Client.`` **JMX MBean** - ``org.apache.cassandra.metrics:type=Client name=`` + ``org.apache.cassandra.metrics:type=Client,name=`` ============================== ================================ =========== Name Type Description ============================== ================================ =========== -ConnectedNativeClients Gauge Number of clients connected to this nodes native protocol server -Connections Gauge> List of all connections and their state information -ConnectedNativeClientsByUser Gauge Number of connnective native clients by username ClientsByProtocolVersion Gauge>> List of up to last 100 connections including protocol version. Can be reset with clearConnectionHistory operation in org.apache.cassandra.db:StorageService mbean. +ConnectedNativeClients Gauge Number of clients connected to this nodes native protocol server +ConnectedNativeClientsByUser Gauge Number of connnective native clients by username +Connections Gauge> List of all connections and their state information RequestsSize Gauge How many concurrent bytes used in currently processing requests RequestsSizeByIpDistribution Histogram How many concurrent bytes used in currently processing requests by different ips ============================== ================================ =========== - Batch Metrics ^^^^^^^^^^^^^ -Metrics specifc to batch statements. +Metrics specific to batch statements. Reported name format: @@ -646,7 +653,7 @@ Reported name format: ``org.apache.cassandra.metrics.Batch.`` **JMX MBean** - ``org.apache.cassandra.metrics:type=Batch name=`` + ``org.apache.cassandra.metrics:type=Batch,name=`` =========================== ============== =========== Name Type Description @@ -656,6 +663,109 @@ PartitionsPerLoggedBatch Histogram Distribution of the number of partiti PartitionsPerUnloggedBatch Histogram Distribution of the number of partitions processed per unlogged batch =========================== ============== =========== +Read Repair Metrics +^^^^^^^^^^^^^^^^^^^ + +Metrics specific to read repair operations. + +Reported name format: + +**Metric Name** + ``org.apache.cassandra.metrics.ReadRepair.`` + +**JMX MBean** + ``org.apache.cassandra.metrics:type=ReadRepair,name=`` + +=========================== ============== =========== +Name Type Description +=========================== ============== =========== +ReconcileRead Meter The rate of read-only read repairs, which do not mutate the replicas +RepairedBlocking Meter The rate of blocking read repairs +SpeculatedRead Meter The rate of speculated reads during read repairs +SpeculatedWrite Meter The rate of speculated writes during read repairs +=========================== ============== =========== + +Messaging Metrics +^^^^^^^^^^^^^^^^^ + +Metrics for internode messaging. + +Reported name format: + +**Metric Name** + ``org.apache.cassandra.metrics.Messaging.`` + +**JMX MBean** + ``org.apache.cassandra.metrics:type=Messaging,name=`` + +=========================== ============== =========== +Name Type Description +=========================== ============== =========== +-Latency Timer Latency of all internode messageing between this node and the datacenters. +-WaitLatency Timer Latency between the message creation time and the time being executed by VERB +CrossNodeLatency Timer Latency of all internode messaging between this node and the peers +=========================== ============== =========== + +Internode Inbound Connection Metrics +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Metrics specific to inbound connections of internode messaging. + +Reported name format: + +**Metric Name** + ``org.apache.cassandra.metrics.InboundConnection..`` + +**JMX MBean** + ``org.apache.cassandra.metrics:type=InboundConnection,scope=,name=`` + +=========================== ============== =========== +Name Type Description +=========================== ============== =========== +CorruptFramesRecovered Guage Estimated number of corrupted frames recovered +CorruptFramesUnrecovered Guage Estimated number of corrupted frames unrecovered +ErrorBytes Guage Estimated number of error bytes +ErrorCount Guage Estimated number of error count +ExpiredBytes Guage Estimated number of expired bytes +ExpiredCount Guage Estimated number of expired count +ScheduledBytes Guage Estimated number of bytes that are pending execution +ScheduledCount Guage Estimated number of message that are pending execution +ProcessedBytes Guage Estimated number of bytes processed +ProcessedCount Guage Estimated number of messages processed +ReceivedBytes Guage Estimated number of bytes received +ReceivedCount Guage Estimated number of messages received +ThrottledCount Guage Estimated number of messages throttled +ThrottledNanos Guage Estimated duration of throttling in nanoseconds +=========================== ============== =========== + +Internode Outbound Connection Metrics +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Metrics specific to outbound connections of internode messaging. + +Reported name format: + +**Metric Name** + ``org.apache.cassandra.metrics.Connection..`` + +**JMX MBean** + ``org.apache.cassandra.metrics:type=Connection,scope=,name=`` + +======================================================= ============== =========== +Name Type Description +======================================================= ============== =========== +[Large|Small|Urgent]MessagePendingTasks Guage Estimated number of pending (Large|Small|Urgent) messages queued +[Large|Small|Urgent]MessagePendingBytes Guage Estimated number of bytes of pending (Large|Small|Urgent) messages queued +[Large|Small|Urgent]MessageCompletedTasks Guage Estimated number of (Large|Small|Urgent) messages sent +[Large|Small|Urgent]MessageCompletedBytes Guage Estimated number of bytes of (Large|Small|Urgent) messages sent +[Large|Small|Urgent]MessageDroppedTasks Guage Estimated number of dropped (Large|Small|Urgent) messages +[Large|Small|Urgent]MessageDroppedTasksDueToOverload Guage Estimated number of dropped (Large|Small|Urgent) messages due to overload +[Large|Small|Urgent]MessageDroppedBytesDueToOverload Guage Estimated number of bytes of dropped (Large|Small|Urgent) messages due to overload +[Large|Small|Urgent]MessageDroppedTasksDueToTimeout Guage Estimated number of dropped (Large|Small|Urgent) messages due to timeout +[Large|Small|Urgent]MessageDroppedBytesDueToTimeout Guage Estimated number of bytes of dropped (Large|Small|Urgent) messages due to timeout +[Large|Small|Urgent]MessageDroppedTasksDueToError Guage Estimated number of dropped (Large|Small|Urgent) messages due to error +[Large|Small|Urgent]MessageDroppedBytesDueToError Guage Estimated number of bytes of dropped (Large|Small|Urgent) messages due to error +======================================================= ============== =========== JVM Metrics ^^^^^^^^^^^ @@ -669,7 +779,7 @@ BufferPool ``jvm.buffers..`` **JMX MBean** - ``java.nio:type=BufferPool name=`` + ``java.nio:type=BufferPool,name=`` ========================== ============== =========== Name Type Description @@ -686,7 +796,7 @@ FileDescriptorRatio ``jvm.fd.`` **JMX MBean** - ``java.lang:type=OperatingSystem name=`` + ``java.lang:type=OperatingSystem,name=`` ========================== ============== =========== Name Type Description @@ -701,7 +811,7 @@ GarbageCollector ``jvm.gc..`` **JMX MBean** - ``java.lang:type=GarbageCollector name=`` + ``java.lang:type=GarbageCollector,name=`` ========================== ============== =========== Name Type Description @@ -734,7 +844,7 @@ MemoryPool ``jvm.memory.pools..`` **JMX MBean** - ``java.lang:type=MemoryPool name=`` + ``java.lang:type=MemoryPool,name=`` ========================== ============== =========== Committed Gauge Amount of memory in bytes that is committed for the JVM to use