wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,012 Commits 36 Branches 335 Tags 297 MiB
Commit Graph

121 Commits

Author SHA1 Message Date
Chaoguang Lin 93ca130602 Delete the unnecessay endKey field in DDMetricsRef, since each begin key is the previous end key 2020-05-19 13:32:42 -07:00
Chaoguang Lin 117ed14321 Add dd-stats zerobytes const to systemData 2020-05-18 11:52:36 -07:00
Chaoguang Lin f515e2c9d0 Add systemdata for dd-stats 2020-05-18 10:36:34 -07:00
Evan Tschannen 9d5e37f1a8 optimized decodeKeyServersValue on the proxy when loading all key ranges on startup 2020-05-09 16:54:35 -07:00
A.J. Beamon 02307ba7b6 Merge branch 'master' into transaction-tagging 
# Conflicts:
#	fdbclient/DatabaseContext.h
 2020-05-09 07:50:29 -07:00
Evan Tschannen ee0be88b99 consistency check can use a UIDtoTag map at a different version from the keyservers which could result in missing entries 2020-05-08 17:45:06 -07:00
Andrew Noyes f1d9c0cb97 Refactor/cleanup 2020-04-30 00:30:37 +00:00
A.J. Beamon 41c517a5dd Merge branch 'master' into transaction-tagging 
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
 2020-04-27 13:05:24 -07:00
A.J. Beamon 35c18ac60a Improvements to expiration. Encode throttles with auto/manual and priority in the key to support throttling the same tag with different values in these parameters. 2020-04-23 20:50:40 -07:00
A.J. Beamon dfec896438 Enforce a throttle limit. Don't count transaction tags on RK if the proxy has updated us in a while. 2020-04-17 11:48:02 -07:00
A.J. Beamon 78d48a0dad Merge branch 'master' into transaction-tagging 
# Conflicts:
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/Knobs.h
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/fdbclient.vcxproj
#	fdbserver/MasterProxyServer.actor.cpp
 2020-04-17 09:23:18 -07:00
chaoguang b011efa8df Merge remote-tracking branch 'upstream/master' into private-keyspace-framework 2020-04-16 10:58:04 -07:00
chaoguang 65620b00f3 Merge remote-tracking branch 'upstream/master' into private-keyspace-framework 2020-04-13 14:21:07 -07:00
Alex Miller db071256db Merge remote-tracking branch 'upstream/master' into reduce-txn-state-store-size 2020-04-12 18:06:10 -07:00
A.J. Beamon ebeca10bce Change the serialization of tags sent in some messages. Add communication of the sampling rate from cluster to clients. 2020-04-09 16:55:56 -07:00
chaoguang 34cddd675c clang-format 2020-04-08 14:33:41 -07:00
chaoguang 4a9658d6b8 fix issues according to andrew's comments 2020-04-08 13:38:12 -07:00
chaoguang 20b2984841 fix issues according to andrew's comments 2020-04-08 12:43:25 -07:00
A.J. Beamon 36da61dd9c Merge branch 'master' into transaction-tagging 
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/vexillographer/fdb.options
 2020-04-07 21:12:14 -07:00
Jingyu Zhou 60407bdee3 Use LiteralStringRef for backup paused key 2020-04-07 16:02:25 -07:00
Jingyu Zhou 9fb3fb9d82 Add pause/resume for new backups 
To pause/resume the backup workers, the fdbbackup command will write to the
backupPausedKey. Then backup workers noticed the value of the key has been
changed and stops/resumes pulling from TLog.
 2020-04-06 14:29:46 -07:00
Alex Miller 6078fd1b18 Convert UID to Tag in keyServers to reduce txnStateStore size 2020-04-05 14:30:09 -07:00
chaoguang b76baa9cc2 Update conflicting keys related constants 2020-04-03 16:08:57 -07:00
A.J. Beamon 2336f073ad Checkpointing a bunch of work on throttles. Rudimentary implementation of auto-throttling. Support for manual throttling via fdbcli. Throttles are stored in the system keyspace. 2020-04-03 15:24:14 -07:00
Andrew Noyes 289487559d Revert "Revert "Merge pull request #2257 from zjuLcg/report-conflicting-key"" 
This reverts commit 804fe1b22e.
 2020-03-24 18:11:15 -07:00
Balachandar Namasivayam 804fe1b22e Revert "Merge pull request #2257 from zjuLcg/report-conflicting-key" 
This reverts commit 648dc4a933, reversing
changes made to 487d131b38.
 2020-03-19 21:34:28 -07:00
chaoguang 0094293d50 add const vars 2020-03-11 23:11:49 -07:00
chaoguang d1c56d3b57 add constant KeyRefs in SystemData 2020-03-11 12:25:50 -07:00
chaoguang 7a76e9556d Merge remote-tracking branch 'upstream/master' into report-conflicting-key 2020-03-04 11:24:39 -08:00
Meng Xu a12a161fb3 Merge branch 'master' into mengxu/fast-restore-pipeline-PR 2020-02-18 14:49:52 -08:00
Jingyu Zhou 5a602f58e8 Start backup with a wait on all backup workers running 
This wait is to make sure that backup workers are already saving mutations so
that no mutations are missed. The idea is that the CLI sets a "backupStartedKey"
in the database and waits for allWorkerStarted() key of the backup to be set.

Backup workers monitor the changes to the "backupStartedKey" and start logging
mutations. Additionally, backup worker for Tag(-2,0) monitors all other workers
have started (checking their saved progress version is larger than the backup's
start version), and then sets the allWorkerStarted() key for the backup.
 2020-01-31 19:29:09 -08:00
Meng Xu 16f9ec45bd Merge branch 'master' into mengxu/fast-restore-pipeline-PR 2020-01-23 20:15:21 -08:00
Jingyu Zhou c08a192c75 Add a backup start key 
If the backup key is not set, do not recruit backup workers for old epoches.
 2020-01-22 19:42:13 -08:00
Jingyu Zhou 19d6a889ff Recruit backup workers for old epochs 
If there are unfinished ranges in the old epochs, the new master will recruit
backup workers responsible for finishing these ranges. These workers remains in
the cluster until the next epoch, when it will remove itself.
 2020-01-22 19:38:45 -08:00
Jingyu Zhou 41f0cf2bb5 Add decode function for backup progress 2020-01-22 19:38:45 -08:00
Jingyu Zhou 7da9f47f26 Enable pop from backup workers 
This is still WIP as some edge cases can trigger test failure, most likely due
to not popping mutations by backup workers when epoch ends.
 2020-01-22 19:38:45 -08:00
Meng Xu bfbf2164c4 FastRestore:Applier buffer data for multiple batches 2020-01-17 17:01:01 -08:00
chaoguang 10719200c3 A hack way to call API through getRange("\xff\xff/conflicting_keys\<start_key>", "\xff\xff/conflicting_keys\<end_key>"). 2020-01-06 11:22:11 -08:00
negoyal d46c7ded59 Merge remote-tracking branch 'origin/master' into storage-cache-subfeature1 2019-11-14 17:52:22 -08:00
negoyal a4a0bf18f9 Merging with Master. 2019-11-12 13:01:29 -08:00
Meng Xu 58aa6711e4 FastRestore:ApplyToDB:BugFix:Serialize integer as bigEndian to ensure lexico order 2019-11-03 17:26:07 -08:00
Andrew Noyes b7b5d2ead3 Remove several nonsensical const uses 
These seem to be all the ones that clang's -Wignored-qualifiers
complains about
 2019-10-26 14:30:34 -07:00
Andrew Noyes de8921b660 Move RestoreWorkerInterface to fdbclient 2019-10-25 10:42:22 -07:00
Andrew Noyes d4de608bb6 Fix OPEN_FOR_IDE build 2019-10-25 10:42:22 -07:00
Jon Fu f4237ebfff Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed 2019-10-16 11:32:16 -07:00
Meng Xu 84b5a5525f FastRestore:Add restoreApplierKeys 2019-10-10 17:18:34 -07:00
Jon Fu 471e283128 Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed 2019-09-18 11:49:07 -07:00
Meng Xu d160810662 FastRestore:Resolve review comments 2019-09-04 16:48:43 -07:00
Jon Fu c908c6c1db added command to fdbcli and changes to SystemData and ManagementAPI 2019-08-27 14:39:43 -07:00
Meng Xu 7ff46e6772 Merge branch 'master' into mengxu/performant-restore-PR 2019-08-07 20:31:56 -07:00