1. Review memory use cases and improve:
Ensure state varialble is initialized and
change unnecessary state variable to variable.
2. Remove debug code that is no longer useful;
3. Mute verbose debug.
1) Use map iterator instead of pointer to maintain stability when map is inserted or deleted
2) dummySampleWorkload: clear rangeToApplier data in each sampling phase. otherwise, we can
have an increasing number of keys assigned to the applier.
1) Should recruit only configured number of roles;
2) Should never register a restore master interface as a restore worker (loader or applier) interface.
1) Use the runRYWTransaction for simple DB access
2) Replace some printf with TraceEvent
3) Remove printf not used in debugging
4) Avoid wait inside the condition in loop-choose-when for
the core routine of restore worker, loader and applier.
5) Rename Restore.actor.cpp to RestoreWorker.actor.cpp since
the file only has functionalities related to restore worker.
Passed correctness test
Add a NotifiedVersion into an applier data which represents
the smallest version the applier is at.
When a loader sends mutation vector to appliers, it sends
the request that contains prevVersion and commitVersion.
This commits also put actor into an actorCollector for
loop-choose-when situation.
Add .h and .cpp files for RestoreLoader and RestoreApplier roles.
We will split the code for each restore role into a separate file.
This commit also fixes the bug in including RestoreCommon.actor.h, and
remove the unused code.
When two struct have the same name but never used in the same scope,
the compiler will NOT report any error in compilation, but
the program will arbitrarily choose one of the struct at the linker time,
and experience weird error in running time. The runtime error is caused by the
corrrupted memory when we assign a struct content to a different struct type.
Each version batch contains at least 0.01MB data
The version batch boundary cannot be within log file's version ranges
Must make sure workers (loaders and appliers) do not exit after applying one batch so that workers can keep working on the next version batch.
Make sure workers and master reset the global variable at the beginning of each version batch.
We use watch to let processes know if the restore request is ready and if the restore has been finished.
When we setup the watch and wait on the watch on a key, we ASSUME that the key has not been set yet.
However, under certain situations (e.g., the restore is fast and the restore request agent is slow), the trigger key may have been set before we wait on the watch.
Without handling this situation, the system will stuck in waiting on the watch.
To solve this situation, we need to check the existance of the key (the watch is on) before we wait on the watch
add data structure to track the status of each node
add logic to let master node assign role to loader and applier
make sure the command request and reply is correct
Copy BackupCorrectness workload to BackupAndParallelRestoreCorrectness workload
Reuse the existing backup and restore code.
The workload is running and can test the restore code.
Next step: Change the restore code to parallel restore code
Meng Xu