b011efa8df
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-04-16 10:58:04 -07:00
65620b00f3
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-04-13 14:21:07 -07:00
c8ab69cce6
Fix incorrect if/ASSERT logic from last minute cleanups.
...
My test code serialized UIDs and Tags, so that I could compare and
verify I got the same thing, and I undid this right before committing,
but forgot to also change the `if` to not rely on UIDs being first also.
2020-04-12 19:38:51 -07:00
db071256db
Merge remote-tracking branch 'upstream/master' into reduce-txn-state-store-size
2020-04-12 18:06:10 -07:00
34cddd675c
clang-format
2020-04-08 14:33:41 -07:00
4a9658d6b8
fix issues according to andrew's comments
2020-04-08 13:38:12 -07:00
20b2984841
fix issues according to andrew's comments
2020-04-08 12:43:25 -07:00
60407bdee3
Use LiteralStringRef for backup paused key
2020-04-07 16:02:25 -07:00
9fb3fb9d82
Add pause/resume for new backups
...
To pause/resume the backup workers, the fdbbackup command will write to the
backupPausedKey. Then backup workers noticed the value of the key has been
changed and stops/resumes pulling from TLog.
2020-04-06 14:29:46 -07:00
6078fd1b18
Convert UID to Tag in keyServers to reduce txnStateStore size
2020-04-05 14:30:09 -07:00
b76baa9cc2
Update conflicting keys related constants
2020-04-03 16:08:57 -07:00
81dd97d27f
Change special keyspace to \xff\xff, \xff\xff\xff
2020-03-31 19:06:45 -07:00
60f6edc3b5
Merge pull request #2860 from zjuLcg/report-conflicting-key-roll-forward
...
Report conflicting key roll forward
2020-03-30 17:33:56 -07:00
113d0fb48b
Remove incorrect assertion
2020-03-27 12:13:30 -07:00
289487559d
Revert "Revert "Merge pull request #2257 from zjuLcg/report-conflicting-key""
...
This reverts commit 804fe1b22e
2020-03-24 18:11:15 -07:00
804fe1b22e
Revert "Merge pull request #2257 from zjuLcg/report-conflicting-key"
...
This reverts commit 648dc4a933487d131b38
2020-03-19 21:34:28 -07:00
0094293d50
add const vars
2020-03-11 23:11:49 -07:00
d1c56d3b57
add constant KeyRefs in SystemData
2020-03-11 12:25:50 -07:00
7a76e9556d
Merge remote-tracking branch 'upstream/master' into report-conflicting-key
2020-03-04 11:24:39 -08:00
a12a161fb3
Merge branch 'master' into mengxu/fast-restore-pipeline-PR
2020-02-18 14:49:52 -08:00
c603b20e7e
FastRestore:Resolve review comments
2020-02-18 14:08:27 -08:00
5a602f58e8
Start backup with a wait on all backup workers running
...
This wait is to make sure that backup workers are already saving mutations so
that no mutations are missed. The idea is that the CLI sets a "backupStartedKey"
in the database and waits for allWorkerStarted() key of the backup to be set.
Backup workers monitor the changes to the "backupStartedKey" and start logging
mutations. Additionally, backup worker for Tag(-2,0) monitors all other workers
have started (checking their saved progress version is larger than the backup's
start version), and then sets the allWorkerStarted() key for the backup.
2020-01-31 19:29:09 -08:00
141609e80a
FastRestore:Improve code style and fix typos
2020-01-27 18:13:14 -08:00
16f9ec45bd
Merge branch 'master' into mengxu/fast-restore-pipeline-PR
2020-01-23 20:15:21 -08:00
1eaea91cb3
Address review comments
2020-01-22 19:42:13 -08:00
c08a192c75
Add a backup start key
...
If the backup key is not set, do not recruit backup workers for old epoches.
2020-01-22 19:42:13 -08:00
19d6a889ff
Recruit backup workers for old epochs
...
If there are unfinished ranges in the old epochs, the new master will recruit
backup workers responsible for finishing these ranges. These workers remains in
the cluster until the next epoch, when it will remove itself.
2020-01-22 19:38:45 -08:00
41f0cf2bb5
Add decode function for backup progress
2020-01-22 19:38:45 -08:00
7da9f47f26
Enable pop from backup workers
...
This is still WIP as some edge cases can trigger test failure, most likely due
to not popping mutations by backup workers when epoch ends.
2020-01-22 19:38:45 -08:00
bfbf2164c4
FastRestore:Applier buffer data for multiple batches
2020-01-17 17:01:01 -08:00
10719200c3
A hack way to call API through getRange("\xff\xff/conflicting_keys\<start_key>", "\xff\xff/conflicting_keys\<end_key>").
2020-01-06 11:22:11 -08:00
d46c7ded59
Merge remote-tracking branch 'origin/master' into storage-cache-subfeature1
2019-11-14 17:52:22 -08:00
a4a0bf18f9
Merging with Master.
2019-11-12 13:01:29 -08:00
630c29d160
FastRestore:resolve review comments
...
1) wait on whenAtLeast;
2) Put BigEndian64 into the function call and the decoder to prevent
future people from making the same mistake.
2019-11-11 17:00:16 -08:00
eb67886b75
FastRestore:Move comment to func definition
...
Resolve review comments.
2019-11-11 15:10:27 -08:00
58aa6711e4
FastRestore:ApplyToDB:BugFix:Serialize integer as bigEndian to ensure lexico order
2019-11-03 17:26:07 -08:00
b7b5d2ead3
Remove several nonsensical const uses
...
These seem to be all the ones that clang's -Wignored-qualifiers
complains about
2019-10-26 14:30:34 -07:00
f4237ebfff
Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed
2019-10-16 11:32:16 -07:00
71509a5157
FastRestore:Applier:applyToDB:Clang format
2019-10-10 17:36:38 -07:00
84b5a5525f
FastRestore:Add restoreApplierKeys
2019-10-10 17:18:34 -07:00
471e283128
Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed
2019-09-18 11:49:07 -07:00
8fbd90e2f6
Merge pull request #1985 from xumengpanda/mengxu/storage-engine-switch-PR-v2
...
Graceful storage engine migration
2019-09-09 13:51:53 -07:00
c2355f721e
Merge branch 'master' into mengxu/performant-restore-PR
2019-09-04 17:11:42 -07:00
d160810662
FastRestore:Resolve review comments
2019-09-04 16:48:43 -07:00
c908c6c1db
added command to fdbcli and changes to SystemData and ManagementAPI
2019-08-27 14:39:43 -07:00
e6284684f0
StorageEngineSwitch:Always remove wrong storeType SS
...
In the old logic of switching storage engines, it marks a storage server
with wrong store type as undesired even though this can lead to no healthy team.
In the first version of the new storage engine switch, we mimic the same logic
of the old version.
2019-08-13 14:59:46 -07:00
a588710376
StorageEngineSwitch:Graceful switch
...
When fdbcli change storeType for storage engines,
we switch the store type of storage servers one by one gracefully.
This avoids recruiting multiple storage servers on the same process,
which can cause OOM error.
2019-08-12 17:37:52 -07:00
4a63de16e9
Merge pull request #1945 from xumengpanda/mengxu/tLog-code-read-v2
...
Add comments to DiskQueue and tLog
2019-08-08 13:24:32 -07:00
7ff46e6772
Merge branch 'master' into mengxu/performant-restore-PR
2019-08-07 20:31:56 -07:00
ba54508c47
code cleanup
2019-08-06 16:30:30 -07:00
chaoguang