wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,969 Commits 36 Branches 335 Tags 297 MiB
Commit Graph

422 Commits

Author SHA1 Message Date
Meng Xu c0c15130b8
Merge pull request #3172 from jzhou77/backup-fix 
Limit memory usage of backup workers
 2020-05-20 15:46:11 -07:00
Steve Atherton 4a827c9304 Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-redwood 2020-05-19 03:08:26 -07:00
Steve Atherton e8626724f9 More refactor of Redwood metrics. Added per-level BTree stats. Added Pager total physical disk reads/writes, and uncacheable read hit/miss counts. Added logging to TraceEvents. 2020-05-16 02:51:57 -07:00
Jingyu Zhou 89ae1200dd Increase backup worker memory buffer size in simulation 
Lower limit causes backup worker to become stuck.
 2020-05-15 09:45:52 -07:00
Steve Atherton 32f4639168 Refactored page remap removal. The process is now called remap cleanup, does batches of remap entries in parallel, coalesces remaps of the same page ID to skip unnecessary writes, and has some knobs for controlling it. FIFOQueue now has peek() to support remap cleanup version lag limits. Added counters for remap cleanup and lazy subtree deletion. Refactored Redwood counters, normalized and grouped their names. 2020-05-15 02:10:51 -07:00
Jingyu Zhou 17915e13b0 Limit memory usage of backup workers 2020-05-14 13:24:56 -07:00
Steve Atherton 421a6581c1 Lazy delete cycles now run more often and not just when the btree is having mutations actively merged into it. Specifically, it is launched upon recovery, then stopped and relaunched after every commit. This will make better use of I/O in between calls to commit(). Also added knobs to control lazy delete cycle parallelism and work limits. 2020-05-13 02:27:03 -07:00
Alex Miller 283fd3af27 Add a knob which controls writing prefix compressed kvs mem snapshots. 
Which will be set to on by default in 7.0
 2020-05-12 17:01:52 -07:00
Evan Tschannen a8e0f1d581 removed knob 2020-05-11 18:20:46 -07:00
Evan Tschannen d0b414ddf2
Merge pull request #3121 from etschannen/master 
Added a large random delay on failure detection
 2020-05-10 17:59:51 -07:00
Evan Tschannen b1bd5ef83e
Merge pull request #3120 from satherton/feature-redwood 
Redwood read concurrency limit, some knobs, and memory-only Pager mode.
 2020-05-10 17:34:41 -07:00
Evan Tschannen 07111f0e41 add a large random delay on failure detection so that not all storage servers need to attempt to become the cluster controller 2020-05-10 17:09:33 -07:00
Steve Atherton 43f9e4dfad Implemented concurrent read limit in IKeyValueStore interface for Redwood. Added knobs for Redwood page size, concurrent read limit, and page fill factor. Changed commitSubtree() recursion back to use a vector and waitForAll() because it seems to be lower overhead than ActorCollection. 2020-05-10 16:13:22 -07:00
Evan Tschannen f9518c3441
Merge pull request #3069 from alexmiller-apple/tls-connection-count 
YOLO at reducing TLS connection count via doing monitorLeader on coordinators
 2020-05-09 17:12:27 -07:00
Evan Tschannen 69affebe40 merge master 2020-05-09 13:29:18 -07:00
Evan Tschannen 2dfae85dc7 the delay for reads is about 15% of the total cost of the read, so start multiple reads with the same delay 2020-05-09 13:26:38 -07:00
A.J. Beamon 02307ba7b6 Merge branch 'master' into transaction-tagging 
# Conflicts:
#	fdbclient/DatabaseContext.h
 2020-05-09 07:50:29 -07:00
Evan Tschannen f0f52fb2be Merge branch 'master' into feature-small-endpoint 
# Conflicts:
#	fdbclient/StorageServerInterface.h
 2020-05-08 16:37:35 -07:00
Jingyu Zhou a833724322
Merge pull request #3078 from xumengpanda/mengxu/fr-circus-stall-PR 
Performant restore: Various improvements based on circus test
 2020-05-07 20:07:23 -07:00
Meng Xu a93c23d239 Resovle review comments 2020-05-07 15:06:59 -07:00
A.J. Beamon fbf436f45f Various cleanup and knob adjustments. 2020-05-07 09:15:33 -07:00
Alex Miller 8a6e177950 Merge remote-tracking branch 'upstream/master' into tls-connection-count 2020-05-05 16:49:36 -07:00
A.J. Beamon b1055a8501 Merge branch 'master' into transaction-tagging 2020-05-05 16:03:39 -07:00
Evan Tschannen f329164fb4
Merge pull request #2532 from dongxinEric/feature/hot-read-key-detection-part-2 
Feature/hot read key detection part 2
 2020-05-05 14:33:34 -07:00
Meng Xu c49b6756fe FastRestoreApplier:Trace clear range op when it has too many for debug 2020-05-05 09:28:50 -07:00
Meng Xu 2fec56e7e2 FastRestore:Logging for getReplyBatches 2020-05-04 20:12:59 -07:00
A.J. Beamon cd2176ac15 Adjust some knobs 2020-05-04 10:37:53 -07:00
A.J. Beamon 36454bb3b8 Merge branch 'master' into transaction-tagging 
# Conflicts:
#	fdbclient/MasterProxyInterface.h
#	fdbclient/NativeAPI.actor.cpp
 2020-05-04 10:23:25 -07:00
A.J. Beamon decf3e82b0 Fix various bugs and make sure to cleanup throttles from the database when they expire 2020-05-01 21:36:28 -07:00
Alex Miller 43a63452d8 YOLO at reducing TLS connection count via doing monitorLeader on coordinators 2020-05-01 14:40:21 -07:00
Evan Tschannen 8520c82ca0 updated dbinfo broadcast to minimize the number of possible WAN round trips 2020-05-01 13:20:26 -07:00
Meng Xu 28178f356f FastRestore:Minor knob change and revise comments 2020-05-01 10:47:44 -07:00
Meng Xu 2c17fff6bb FastRestore:Add batchIndex into metric event name 2020-05-01 10:05:33 -07:00
Meng Xu 038f3834fc Merge branch 'master' into mengxu/fr-code-improvement-PR 2020-05-01 09:26:29 -07:00
Jingyu Zhou b3698ac921
Merge pull request #3060 from xumengpanda/mengxu/fr-agent-fix-PR 
Fix fastrestore_agent errors when run in non-simulation mode
 2020-04-30 22:46:39 -07:00
Meng Xu fb1c456a2d FastRestore:Change default knob value 2020-04-30 19:15:32 -07:00
A.J. Beamon 66228343f1 Merge branch 'master' into transaction-tagging 2020-04-30 08:12:03 -07:00
Meng Xu a0d67cac16 Merge branch 'master' into mengxu/fr-code-improvement-PR 2020-04-29 21:07:33 -07:00
A.J. Beamon 6ada5359b8 Merge branch 'master' into transaction-tagging 2020-04-29 14:27:21 -07:00
Evan Tschannen 7cebe743f9 A number of bug fixes of rare correctness errors 2020-04-29 13:50:13 -07:00
Evan Tschannen 76fb345dd1 Merge branch 'master' into feature-tree-broadcast 
# Conflicts:
#	fdbrpc/FailureMonitor.actor.cpp
 2020-04-29 09:51:22 -07:00
Meng Xu 8dd05405eb FastRestore:Guard knob with BUGGIFY 
Prevent knob to be randomly set.
 2020-04-28 16:05:40 -07:00
A.J. Beamon b80225dde0 Initial support for ramping load back up. Fix some logging. Update auto-throttles less frequently. 2020-04-28 15:50:45 -07:00
A.J. Beamon 0ed70accfa Reorganization of throttle storage in ratekeeper to support various auto-throttling related actions 2020-04-28 14:30:37 -07:00
Evan Tschannen b7f5f3be48 merge in master 2020-04-28 13:11:47 -07:00
A.J. Beamon 41c517a5dd Merge branch 'master' into transaction-tagging 
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
 2020-04-27 13:05:24 -07:00
Meng Xu ee5051792a Add BUGGIFY back to DESIRED_TEAMS_PER_SERVER knob 
Kudos to A.J. who found this bug.
 2020-04-24 12:54:31 -07:00
A.J. Beamon 239876351b Add some initial auto-throttling. Move the definition of the priority enum to a more global place and use it for all transaction priorites (except in ClientLogEvents, because of serialization incompatibilites). 2020-04-24 11:31:16 -07:00
Evan Tschannen c87aa33941 Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	bindings/go/src/fdb/generated.go
#	documentation/sphinx/source/api-common.rst.inc
#	documentation/sphinx/source/api-ruby.rst
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/FailureMonitorClient.actor.cpp
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/vexillographer/fdb.options
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/OldTLogServer_6_0.actor.cpp
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	versions.target
 2020-04-23 13:47:53 -07:00
Evan Tschannen 0c84ad4bc6
Merge pull request #2917 from bnamasivayam/fail-slow-ss 
Mark the storage servers that are continually lagging as unhealthy
 2020-04-22 23:18:35 -07:00