wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,033 Commits 36 Branches 335 Tags 297 MiB
Commit Graph

266 Commits

Author SHA1 Message Date
Evan Tschannen 3f9d9d8b84 Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	cmake/FlowCommands.cmake
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/StorageServerInterface.h
#	fdbserver/DataDistributionTracker.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/Knobs.h
#	flow/Platform.cpp
#	versions.target
 2020-01-16 18:37:47 -08:00
Evan Tschannen 4b90487b90 occasionally throw wrong_shard_server when waitMetrics times out so that the waitMetrics request can get the correct number of shards if two shards have been merged or split and the same storage server owns all the chunks 2020-01-15 13:22:18 -08:00
A.J. Beamon 9668c4471f Clamp infinite limit in ratekeeper 2020-01-14 15:45:24 -08:00
Evan Tschannen 17e97f24e4
Merge pull request #2526 from etschannen/feature-dd-improvements 
Data distribution improvements
 2020-01-10 17:53:22 -08:00
Evan Tschannen fde53cbeef HasBeenTrueFor was ready immediately after a previous shard merge 2020-01-10 16:28:56 -08:00
Evan Tschannen 9b80498180 Added a trace event to warn if a shard is merged before enough time has elapses from becoming low bandwidth 2020-01-10 14:58:38 -08:00
Evan Tschannen 4aab9b7bc8 fix: clients would waste time attempting to read from a remote region when it was in the process of catching up 2020-01-10 12:23:59 -08:00
Evan Tschannen 02a8e8d1e9 batch priority must be heavily throttled before stopping data distribution rebalancing 2020-01-09 17:05:22 -08:00
Evan Tschannen 9842272ced raised the priority of shard merges, because the tracker cannot track an unmerged shard 2020-01-09 17:04:17 -08:00
Evan Tschannen e4fa4ad0c9 Data distribution will not merge a shard unless it has been low bandwidth for 5 minutes 2020-01-09 17:02:49 -08:00
Evan Tschannen ab7071932f Data distribution no longer attempts to pick teams which share members of the source unless the team matches exactly 2020-01-09 16:59:37 -08:00
Meng Xu 39a4f2372f Change FASTRESTORE_SAMPLING_PERCENT to 0 to 100 2019-12-04 21:26:27 -08:00
Meng Xu c6b36dbffb FastRestore:Sampling:Resolve review comments 2019-12-04 17:35:11 -08:00
Meng Xu dd91d26dfa FastRestore:Sampling:Add FASTRESTORE_SAMPLING_RATE knob 2019-12-04 11:46:29 -08:00
Evan Tschannen 07331ab5fd
Merge pull request #2362 from etschannen/master 
Merge 6.2 into master
 2019-12-02 15:04:27 -08:00
Evan Tschannen ebcb2f79ed Merge branch 'master' of github.com:apple/foundationdb 2019-11-22 15:34:49 -08:00
Xin Dong 14dd5626d7 Resolve review comments 2019-11-22 10:11:45 -08:00
Xin Dong b6e1839d84 Code clean up 2019-11-21 13:39:19 -08:00
Xin Dong b282e180d5 Added a knob to disable read sampling 2019-11-20 14:03:20 -08:00
Evan Tschannen 8d3ef89540 Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/MutationList.h
#	fdbserver/MasterProxyServer.actor.cpp
#	versions.target
 2019-11-14 15:49:56 -08:00
negoyal a4a0bf18f9 Merging with Master. 2019-11-12 13:01:29 -08:00
Evan Tschannen 396dccbc98 when peeking from satellites we do not need to limit the amount of peeking on log router tags, because that is the only thing that can be peeked from a satellite log 2019-11-08 18:34:05 -08:00
Evan Tschannen afc9713005 Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/FDBTypes.h
#	fdbserver/LogSystem.h
#	fdbserver/LogSystemPeekCursor.actor.cpp
#	fdbserver/OldTLogServer_6_0.actor.cpp
#	fdbserver/TLogServer.actor.cpp
#	versions.target
 2019-11-06 13:45:37 -08:00
Evan Tschannen daac8a2c22 Knobified a few variables 2019-11-04 20:21:38 -08:00
Evan Tschannen 71dfaa3f95
Merge pull request #2275 from dongxinEric/bugfix/2273/fix-read-key-sampling 
Resolves #2273: Use a large value for read sampling size threshold. Also at sampling …
 2019-10-31 10:21:58 -07:00
Xin Dong 199a34b827 Defined a minimum read cost (a penalty) for empty read or read size smaller than it. Fixed several review comments. 2019-10-30 10:04:19 -07:00
Evan Tschannen 3325980c03 Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/OldTLogServer_6_0.actor.cpp
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/WorkerInterface.actor.h
#	fdbserver/worker.actor.cpp
#	versions.target
 2019-10-24 17:38:15 -07:00
Xin Dong a290e2cb2b Use 8 MiB for real 2019-10-24 11:02:17 -07:00
Xin Dong fe54a4bde1 - Changed SHARD_MAX_BYTES_READ_PRE_KEYSEC to be equivalent to 8MiB/s, which when times the sample expire interval(120 seconds) yields 960MiB/s. A shard having a read rate larger than that will be marked as read-hot. The number 960MiB was chosen to be roughtly twice the size of the max allowed shard size to avoid wrongly marking a shard as read-hot when doing a table scan on it. 
- Also tuned down the empty key sampling percentage to be 5%.
 2019-10-23 12:00:19 -07:00
Meng Xu e676348710
Merge pull request #1955 from fzhjon/mark-ss-failed 
Add fdbcli and API command to mark storage servers as permanently failed
 2019-10-22 23:36:30 -07:00
Xin Dong af72d15566
Update fdbserver/Knobs.cpp 
From AJ: to match typical aligned format used on other variables.

Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
 2019-10-22 13:53:28 -07:00
Xin Dong e6f5748791 Use a large value for read sampling size threshold. Also at sampling site, don't round up small values to avoid sampling every key. 2019-10-22 13:47:58 -07:00
A.J. Beamon 29a0014b41 Fix "bandwith" typo 2019-10-22 09:51:59 -07:00
Evan Tschannen 12c517ab16 limit the number of committed version updates in progress simultaneously to prevent running out of memory 2019-10-21 16:01:45 -07:00
Xin Dong fca9aab17a
Merge pull request #2046 from dongxinEric/feature/hot-read-key-detection 
Added metrics for read hot key detection
 2019-10-21 14:31:48 -07:00
Jon Fu d2b6626d5c Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed 2019-10-21 13:47:06 -07:00
Evan Tschannen 688940b685 merge 6.2 into master 2019-10-21 11:43:46 -07:00
Evan Tschannen 8b09cd16b2 Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-share-mutations 2019-10-16 14:50:37 -07:00
Evan Tschannen 5667331729 added a buggify + minor code cleanup 2019-10-11 18:31:43 -07:00
Evan Tschannen 86bcb84b45 Raised the data distribution priority of splitting shards above restoring fault tolerance to avoid hot write shards 2019-10-11 17:50:43 -07:00
Xin Dong 62ffdd54a3 Updated some comments to reflect the correct knob value and also used a more appropiate value for read bandwidth. Set the default value for read bandwidth in some cases. 2019-10-09 16:42:42 -07:00
Xin Dong cd4757b06c Address review comments 2019-10-09 16:42:42 -07:00
Xin Dong 6b0f771cc0 Fixex a typo in knobs. Addressed some review comments. Added code for actual metric collecting. 2019-10-09 16:42:42 -07:00
Xin Dong 12293d5497 Added metrics for read hot key detection 2019-10-09 16:42:42 -07:00
Jon Fu d96a7b2c69 Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed 2019-10-03 09:47:45 -07:00
Evan Tschannen 628b4e0220 added a warning if multiple log ranges exist for the same range 2019-10-02 17:06:19 -07:00
Meng Xu d0147e5e5d Merge branch 'release-6.2' into mengxu/merge-release620-to-master-v3 
Resolved Conflicts:
	documentation/sphinx/source/release-notes.rst
	fdbserver/DataDistribution.actor.cpp
	versions.target
 2019-10-02 13:22:56 -07:00
Jon Fu 68f88dea4b remove buggify setting of new knob 2019-09-27 13:12:41 -07:00
Jon Fu 450a09e117 Code Review Changes 2019-09-24 15:48:50 -07:00
Meng Xu d2fd1f4931 DD:MisconfiguredLocality:Fix review comments 2019-09-17 13:04:21 -07:00