A watch created on a transaction must resolve when the creating transaction's
commit fails (per the documented watch contract: "If the transaction used to
create a watch encounters an error during commit, then the watch will be set
with that error."). On the normal commit-error path commitAndWatch() calls
cancelWatches() to do this, but when the commit is cancelled — e.g. by a
transaction timeout firing through resetPromise — commitAndWatch() is itself
cancelled before it reaches cancelWatches(). The watch future is then left
pending forever: the application waits on a watch that will never fire and
never error.
This surfaced in the fdb_c upgrade/wiggle apitester (WatchAndWait workload)
when running the fdb_c_wiggle_only (and related fdb_c_upgrade_*) ctest:
with a transaction timeout configured, a commit that timed out during the
wiggle left its watch orphaned, so the workload stopped making progress and
the post-step progress check failed.
Fix: in RYWImpl::commit's error handler, call cancelWatches(e) (guarded so
that an outer actor_cancelled does not stomp the natural transaction_cancelled
resolution that ~Transaction() would otherwise produce). The call is a no-op
when watches have already been resolved or cleared by commitAndWatch() itself,
and closes the gap on the cancellation/timeout path for both pre-410 and
modern api-version callers.
Known narrow follow-up: ReadYourWritesTransaction::commit() short-circuits
when resetPromise is already set, bypassing RYWImpl::commit entirely. A watch
that has been handed off to the native transaction (RYWImpl::watch line 1327)
in the window between watch registration and the commit call is still left
orphaned in that path. Addressing it requires more care — naively calling
cancelWatches() from the non-actor short-circuit regressed Joshua; deferring
or relocating the call is left for a follow-up.
Test: simulation regression workload (WatchCommitTimeout) arms a watch on a
transaction with a short FDB_TR_OPTION_TIMEOUT and asserts that every watch
on a timed-out commit eventually resolves (orphanedWatches == 0). The workload
does not assert *how* the watch resolves — under cluster recovery the
SS-side watch actor can resolve it independently (transaction_too_old,
process_behind, etc., or even Void) before the commit-error propagation path
reaches the promise; all such outcomes satisfy the no-orphan contract.
Validated: pre-fix produces orphaned watches; post-fix runs many thousands of
timed-out commits with zero orphans, and 100k Joshua passes clean.
Michael Stack