- Distinguish corrupt/incomplete cases; latter case only refuses uncertain data
- Ensures quorum durability before marking own log faulty
- Refuse unsafe operations more selectively once log marked faulty
- Simplify and merge logic with standard bootstrap
- fix race conditions when updating Task.info (between running and Exclusive methods)
- do not run consequences of actions that fail and do not become durable (e.g. applying to data store/post-applying a transaction that we failed to complete the PreApplied/Applying step for)
duplicate MergeIterator class to avoid megamorphic calls in hot path for Row/ComplexCell and UnfilteredRowIterators
disable inlining for sinkInBinaryHeap, move typical case outside to improve hot path inlining
replace List for versions with array
fast path for same column metadata in cell merger
avoid potential megamorphic cell method invocation in ALIVE DeletionTime
minDeletionTime calculation optimization for merge logic
do not add an empty iterator, merge 2 loops into 1
patch by Dmitry Konstantinov; reviewed by Francisco Guerrero for CASSANDRA-21524
Description:
AbstractType.writeValue() is one shared method. All column types use it.
Inside writeValue(), it calls valueLengthIfFixed(). This is a virtual call. Many types override this method (Int32Type, LongType, UTF8Type, ...).
In a real cluster, many column types pass through writeValue(). So the profile always sees lots types.
Because of this, the JIT cannot inline the call. It stays as a vtable call. Also, the compiled body of writeValue() becomes big, so the JIT refuses to inline writeValue() itself ("already compiled into a big method").
We also see the same itable/vtable stubs in async-profiler output from a real cluster, running with default production options and a normal workload.
How to solve:
Add a final int field to AbstractType. Set it in the constructor. writeValue() reads this field instead of calling valueLengthIfFixed().
A field read needs no type profile. So profile pollution has no effect on it. The valueLengthIfFixed() method is not changed. Only the write path uses the field.
Result:
Production is always the polluted state, so this is the real-world comparison
JMH, JDK 17 / arm64, 1 thread, 10 x 1s warmup and measurement:
before after improvement
readValue 49.99M ops/s 56.44M ops/s +12.90%
writeValue 99.63M ops/s 114.46M ops/s +14.88%
patch by Koo Taejin; reviewed by Dmitry Konstantinov, Francisco Guerrero for CASSANDRA-21536
minDeletionTime is also added to Cell to avoid double invocation of localDeletionTime method
patch by Dmitry Konstantinov; reviewed by Francisco Guerrero for CASSANDRA-21526
* Add ephemeral-storage requests and limits to agents, and log peak disk usage
Agents were evicted at the node level due to lack of disk space. Each pod needs minimum 100GiB, and each template now budgets 80Gi of the ~89Gi that a 100GiB node allocates, with docker's own volume carries a 60Gi sizeLimit. Exceeding either evicts the pod, so logging has been added in each test split before and after the run. All script blocks in Jenkinsfile are now labeled for improved readability in test runs.
* `.build/run-ci` must not silently drop a site's helm values customisations
Long-lived instances like pre-ci.cassandra.apache.org carry values in addition to the jenkins-deployment.yaml, and any `helm upgrade -f jenkins-deployment.yaml` (or `.build/run-ci`) would drop them. Allow these values to be kept in a separate file, passed in with `--values-override`. Before upgrading helm values, we now compare the deployed values against what is about to be applied and require confirmation for anything only the site holds. This catches added keys and dropped list items (but it cannot catch every change).
Also keep the jenkins-home pvc claim on uninstall, so --only-tear-down no longer destroys build history. Add jenkins-test.sh and a gha workflow for test validating changes under .jenkins/
patch by Mick Semb Wever; reviewed by Dmitry Konstantinov for CASSANDRA-21523
Fairness: tasks are grouped by kind of work, and for each group we track work arrival and service via decaying counters. If work for some group(s) begin to be served at a much lower rate than some other group (e.g. because that group has a large backlog, that by simple priority comes first), then we begin processing some fraction of the work by service-ratio rather than priority. This means that we cannot starve request processing because, e.g., a sync point has produced thousands of state save tasks.
Incremental processing: tasks that process many keys may be declared INCR or ASYNC. ASYNC indicates the keys are needed (so should be loaded), but may be used in follow-up work so the task may be executed if the data is not yet loaded. An ASYNC task may be followed by an INCR task that processes keys as they are loaded, with limits to the batch sizes. This limits the latency impact of processing larger transactions such as sync points. INCR tasks may be declared to be processed in either an arbitrary order with no isolation, or “atomically” with its parent task. In the latter case, the complete unit of work appears to execute as a single execution to external observers (blocking progress on unprocessed keys).
Key-level ordering: AccordCacheEntry can inflate a special Queue object that can be used to impose ordering constraints: FIFO for atomic work (once it has been part processed, or submitted as part of a parent task); prioritised for tasks that have a key-level priority to impose (i.e. PreAccept/Accept/Commit/Stable/Apply want to be ordered by TxnId); and unsequenced tasks for those that have no sequencing restrictions.
Additional improvements:
- AccordCacheEntry are now explicitly LOCKED for the duration of their usage, which may span multiple executions for long running INCR tasks.
- ExclusiveExecutor releases its ownership lock immediately, since AccordCacheEntry locks guarantee safety (as they will not be released until the task is cleaned up)
- CassandraThread tracks active and locked AccordExecutor, so that we may safely and more aggressively use executeMaybeImmediately, safe in the knowledge it will never permit cross-executor executions or multiple executor locks to be held.
Also:
- Rename PreLoadContext -> ExecutionContext
- Break out AccordExecutor/AccordTask into separate package
A 32-bit length field read from the wire by CBUtil.readValue (and its
siblings) flowed directly into new byte[length] with no upper bound,
allowing an unauthenticated client to drive the JVM into
OutOfMemoryError: 'Requested array size exceeds VM limit' — and, with
the default -XX:OnOutOfMemoryError=kill -9 %p, terminate the process —
by declaring Integer.MAX_VALUE as the SASL-token length in AUTH_RESPONSE.
Guard the allocation in the single private readRawBytes(ByteBuf, int)
that all int32-length readers funnel through, rejecting lengths that
exceed the buffer's readable bytes with a ProtocolException.
patch by Francisco Guerrero; reviewed by Stefan Miklosovic for CASSANDRA-21521
Benedict Elliott Smith