Note: Server processes started getting reported as clients since 7.1.0
(not sure if this change in behavior was intentional or not), and this
breaks the operator upgrade logic.
* Remove boundaryAndExist
Client knows whether an entry is a boundary, and client also can
see whether the record for that entry exist by checking whether
the rangeResult is empty. This field is useless.
This field breaks backward compatibility, need to remove it.
In NativeAPI.actor.cpp, ::watch ACTOR will call cx->clearWatchMetadata()
when the connectionFileChanged event is triggered. After that, it will
create a new watch metadata for itself.
If there are multiple watches, each of them that receives the
cnnectionFileChanged will clear *all* watch data and create *one* watch
for itself. This does not makes sense. A watch should only clear the
metadata, and then create one, for itself.
cx->clearWatchMetadata() is only used there, thus removed.
In the case
1. A watch to key A is set, the watchValueMap ACTOR, noted as X, starts waiting.
2. All watches are cleared due to connection string change.
3. The watch to key A is restarted with watchValueMap ACTOR Y.
4. X receives the cancel exception, and tries to dereference the counter. This causes Y gets cancelled.
the reference count will cause watch prematurely terminate. Recording
the versions of each watch would help preventing this issue
Currently, there is a cyclic reference situation in
DatabaseContext -> WatchMetadata -> watchStorageServerResp ->
DatabaseContext
If there is a watch created in the DatabaseContext, even the
corresponding wait ACTOR is cancelled, the WatchMetadata will still hold
a reference to watchStorageServerResp ACTOR, which holds a reference to
DatabaseContext.
In this situation, any DatabaseContext who held a watch will not be
automatically destructed since its reference count will never reduce to
0 until the watch value is changed. Every time the cluster recoveries,
several watches are created, and when the cluster restarts, the
DatabaseContext which not being used, will not be able to destructed due
to these watches.
With this patch, each wait to the watch will be counted. Either the
watch is triggered or cancelled, the corresponding count will be
reduced. If a watch is not being waited, the watch will be cancelled,
effectively reduce the reference count of DatabaseContext. This will
hopefully fix the issue mentioned above.
The code is tested by 1) Manually change the number of logs of a local
cluster, see the cluster recovery and previous DatabaseContext being
destructed; 2) 100K joshua run, with 1 failure, the same test will fail
on the current git main branch.
* add bytelimit for prefetch
A fraction of byteLimit will be used as the limit to fetch index.
For the indexes fetched, fetch records for them in batch.
byteLimit always count the index size, it also count record if exist,
it at least return 1 index-record entry and always include the last entry
despite that adding the last entry despite it might exceed limit.
There is a Knob STRICTLY_ENFORCE_BYTE_LIMIT, when it is set, records
will be discarded once the byteLimit is hit, despite they are fetched.
Otherwise, return the whole batch.
Currently, blobGranuleGetTenantEntry() expects to get the TenantName
through the transaction. Database calls cannot abide by this contract.
So enable the call to take in a TenantName and expect at least 1 to be
set.
* Account for tenant in during cache invalidation in NativeAPI actor.
* Account for tenant prefix while checking whether a shard is readable
on storage server.
This commit also adds many debug TraceEvents that will later be
removed.
This change makes the getEstimatedRangeSizeBytes function tenant aware.
Previously, this function would return the size of the requested
keyspace even if the tenant in the Transaction or DatabaseContext did
not match the tenant corresponding to the keyspace.
Also make some improvements to the new workload.
Hao Fu