natas_hw
/
cassandra
mirror of https://github.com/apache/cassandra
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

CASSANDRA-20147: Fix Semaphore permit count overflow under sustained high write rate 

In Batch commit mode, if multiple writes arrive during a single commitlog flush
it will release the haveWork semaphore more times than it's acquired.  If this
happens often enough without an idle period, it will eventually overflow.
So, every time we acquire() the permit count should be reset to 0.
This is similar to how it worked in 3.0, but with more places it's acquired.
In theory this leaves a potential race, but only if 2 billion writes arrive within
a single commitlog flush interval.
Without this change, I believe the flusher loop would also run without waiting
for a while during idle periods after sustained high load.

patch by Elliott Sims (elliott@backblaze.com); reviewed by TBD for CASSANDRA-20147
This commit is contained in:
Elliott Sims 2025-01-23 16:49:39 -07:00
parent b4bcdfa785
commit 1f3ceb7253
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
src/java/org/apache/cassandra/db/commitlog/AbstractCommitLogService.java
View File

@ -203,6 +203,7 @@ public abstract class AbstractCommitLogService
if (markerIntervalNanos <= 0)
{
haveWork.acquire(1);
haveWork.drain();
}
else
{
@ -213,6 +214,7 @@ public abstract class AbstractCommitLogService
long wakeUpAt = pollStarted + markerIntervalNanos;
if (wakeUpAt > now)
haveWork.tryAcquireUntil(1, wakeUpAt);
haveWork.drain();
}
}
catch (Throwable t)
@ -221,6 +223,7 @@ public abstract class AbstractCommitLogService
throw new TerminateException();
else // sleep for full poll-interval after an error, so we don't spam the log file
haveWork.tryAcquire(1, markerIntervalNanos, NANOSECONDS);
haveWork.drain();
}
}