1) Remove global map to buffer the parsed mutations on loader.
Use local map instead to increase parallelism.
2) Use std::map<LoadingParam, Future<Void>> to hold the actor
that parse a backup file and to de-duplicate requests.
3) Remove unused code.
1) Add sendBatchRequests and getBatchReplies
sendBatchRequests is a generic actor to send requests without
processing replies.
getBatchReplies is similar to sendBatchRequests expect that
it returns the reply to caller.
2) Share applier interface to loaders by using RequestStream,
instead of using DB.
Create RestoreSysInfo struct, similar purpose as DBInfo, for
the restore system information that are shared among restore workers.
Add a NotifiedVersion into an applier data which represents
the smallest version the applier is at.
When a loader sends mutation vector to appliers, it sends
the request that contains prevVersion and commitVersion.
This commits also put actor into an actorCollector for
loop-choose-when situation.
When mutationVectorThreshold is not 1, a loader sends a vector of
mutations to an applier.
We should never mix mutations at different versions into the same vector.
The code on previous commit may mix mutations at versions.
This commit resolves the bug.
In the sampling phase, a loader will cache the mutations into kvOps map;
In the loading log file phase, the loader will do the same thing.
The loader must clear the kvOps map once the loader use it; otherwise,
it will cache the sampled mutations twice, which leads to an
inconsistent restored DB.
This commit identifies the bug
why DB may be restored to an inconsistent state.
The cmdid is used to achieve exact once delivery even when
network can deliver a request twice.
This is under assumption that cmdid is unique for each request!
However, this assumption may not hold for
the phase Loader_Send_Mutations_To_Applier, when loaders send parsed
mutations to appliers:
1) When the same loader loads multiple files, we reset the cmdid
for the phase;
2) When different loaders load files, each loader's cmdid starts from
0 for the phase.
Both situations can break the assumption, which causes appliers to
miss some mutations to apply. This breaks the cycle test.
The current code uses one restore interface to handle the work
for all restore roles, i.e., master, loader and applier.
This makes it harder to review or maintain or scale.
This commit split the restore into multiple roles by mimicing FDB
transaction system:
1) It uses a RestoreWorker as the process to host restore roles;
This commit assumes one restore role per RestoreWorker; but
it should be easy to extend to support multiple roles per RestoreWorker;
2) It creates 3 restore roles:
RestoreMaster: Coordinate the restore process and send commands to the other two roles;
RestoreLoader: Parse backup files to mutations and send mutations to appliers;
RestoreApplier: Sort received mutations and apply them to DB in order.
Compilable version. To be tested in correctness.
Add .h and .cpp files for RestoreLoader and RestoreApplier roles.
We will split the code for each restore role into a separate file.
This commit also fixes the bug in including RestoreCommon.actor.h, and
remove the unused code.
Meng Xu