Commit Graph

92 Commits

Author SHA1 Message Date
Evan Tschannen c9ee83e1b1 fix: do not buggify PEEK_TRACKER_EXPIRATION_TIME to a value of 20 2021-11-24 11:28:57 -08:00
Steve Atherton 508429f30d
Redwood chunked file growth and low priority IO starvation prevention (#5936)
* Redwood files now growth in large page chunks controlled by a knob to reduce truncate() calls for expansion.   PriorityMultiLock has limit on consecutive same-priority lock release.  Increased Redwood max priority level to 3 for more separation at higher BTree levels.

* Simulation fix, don't mark certain IO timeout errors as injected unless the simulated process has been set to have an unreliable disk.

* Pager writes now truncate gradually upward, one chunk at a time, in response to writes, which wait on only the necessary truncate operations.   Increased buggified chunk size because truncate can be very slow in simulation.

* In simulation, ioTimeoutError() and ioDegradedOrTimeoutError() will wait until at least the target timeout interval past the point when simulation is sped up.

* PriorityMultiLock::toString() prints more info and is now public.

* Added queued time to PriorityMultiLock.

* Bug fix to handle when speedUpSimulation changes later than the configured time.

* Refactored mutation application in leaf nodes to do fewer comparisons and do in place value updates if the new value is the same size as the old value.

* Renamed updatingInPlace to updatingDeltaTree for clarity.  Inlined switchToLinearMerge() since it is only used in one place.

* Updated extendToCover to be more clear by passing in the old extension future as a parameter.  Fixed initialization warning.
2021-11-12 13:47:07 -08:00
Daniel Smith 394b9dc619 Code review changes 2021-11-10 11:53:27 -05:00
Daniel Smith f6342b0a8d Update defaults 2021-11-10 11:51:05 -05:00
Daniel Smith 66520eb1c1 Utilize read types to do selective throttling 2021-11-10 11:51:04 -05:00
Tao Lin fdb3b72e35 Introduce GetRangeAndFlatMap to push computations down to FDB
Re-introduce #5609
2021-11-09 13:52:28 -08:00
Tao Lin 586cc3b102
Revert "Introduce GetRangeAndFlatMap to push computations down to FDB" 2021-11-04 08:46:56 -07:00
Tao Lin 0853661d13 Introduce getRangeAndHop to push computations down to FDB 2021-11-03 13:21:16 -07:00
Xiaoxi Wang 1a2a838df3 add knob 2021-10-27 09:08:37 -07:00
Xiaoxi Wang 69190ed04e format 2021-10-27 09:08:37 -07:00
Xiaoxi Wang 0053b4793e change knob and delete redundant doBuildTeam 2021-10-27 09:08:37 -07:00
Evan Tschannen 2208b04174
Merge pull request #5855 from sfc-gh-etschannen/blob_full_clean
Blob Granules V0
2021-10-26 09:57:35 -07:00
Lukas Joswiak c96f560cbe Verify rollback of a single version in simulation, other small fixes 2021-10-25 12:03:22 -07:00
Josh Slocum 0ff8ddc2b6 Merge branch 'master' into blob_full_clean 2021-10-25 13:38:48 -05:00
Steve Atherton d153519188
Merge pull request #5813 from sfc-gh-jslocum/ss_ebrake_streaming_fix
Fixes to ss e-brake, tlog streaming, and their interaction
2021-10-22 10:46:17 -07:00
Josh Slocum 773886515e Merge branch 'feature-range-feed' into blob_full_clean 2021-10-22 11:07:51 -05:00
Zhe Wu 0cf829ef91 Reduce restore error message 2021-10-20 14:02:48 -07:00
Josh Slocum 8dd7f8f447 Fixes to ss e-brake, tlog streaming, and their interaction 2021-10-20 10:48:29 -05:00
Suraj Gupta ff0d687704 Cleanup comments for server knobs. 2021-10-18 17:45:30 -04:00
A.J. Beamon 507a09893c
Add ClientCount to ClusterControllerMetrics (#5748) 2021-10-17 20:47:11 -07:00
Josh Slocum 5f0ec0612a Merge branch 'feature-range-feed' into blob_full 2021-10-13 15:44:35 -05:00
Suraj Gupta 2ec8781224 Merge knobs into one. 2021-10-13 14:00:37 -04:00
Suraj Gupta a163619fbc Change default val for knob. 2021-10-13 09:58:09 -04:00
Suraj Gupta 5a6a052c55 Add a knob to gate blob-related work. 2021-10-13 09:48:02 -04:00
Zhe Wu c0fbe5471f Implement the core logic of grey failure triggered failover 2021-10-07 11:19:34 -07:00
Suraj Gupta 4d54669ccd Recruit the blob workers via blob manager.
In this PR, the blob manager now recruits blob workers
(via communication with the cluster controller). Blob workers
are onboarded as blob worker processes enter the cluster.
2021-10-04 11:07:08 -04:00
Zhe Wu cb154c12d4 disable ENABLE_DETAILED_TLOG_POP_TRACE by default 2021-09-26 21:46:57 -07:00
A.J. Beamon c2885ab70d The BACKUP_LOCK_BYTES knob could be buggified to a value that was too low, resulting in backup getting stuck. 2021-09-24 09:15:30 -07:00
Suraj Gupta 5fa6c687d6 Add blob manager as a singleton. 2021-09-23 10:45:37 -04:00
Suraj Gupta 72edcd8d73 Address PR comments.
Revert knob name change, fix comparison between new and old
recruitments, and get rid of empty `if` block.
2021-09-22 16:56:34 -05:00
Suraj Gupta 0b6fecddbc Refactor logic for recruiting singletons.
This commit refactors the logic for recruiting singletons,
which is done by the ClusterController. This allows for far
easier additions of new singletons in the future, and also
cleans up the code.

Also, the logic for recruiting DD was changed to mirror
the logic for recruiting RK. Although the logic for RK
allows there to be many RKs existing at once, the moveKeysLock
mechanism used by DD still prevents multiple DDs existing at once.
2021-09-22 16:56:18 -05:00
Suraj Gupta 6533678f0d Address PR comments.
Revert knob name change, fix comparison between new and old
recruitments, and get rid of empty `if` block.
2021-09-20 14:26:42 -05:00
Suraj Gupta fe098b3b11 Refactor logic for recruiting singletons.
This commit refactors the logic for recruiting singletons,
which is done by the ClusterController. This allows for far
easier additions of new singletons in the future, and also
cleans up the code.

Also, the logic for recruiting DD was changed to mirror
the logic for recruiting RK. Although the logic for RK
allows there to be many RKs existing at once, the moveKeysLock
mechanism used by DD still prevents multiple DDs existing at once.
2021-09-20 14:26:42 -05:00
Josh Slocum c9b8bdffbe Passes a single correctness test! 2021-09-15 17:18:04 -05:00
Josh Slocum c2d1d1704f Merge branch 'feature-range-feed' into blob_full 2021-09-10 11:21:52 -05:00
Josh Slocum eb76343dfb Added blob granule reassignment and splitting 2021-09-08 14:09:14 -05:00
Steve Atherton be440ab954
Merge pull request #5260 from FuhengZhao/RedwoodHistogram
Redwood local histograms
2021-08-26 12:05:44 -07:00
Josh Slocum 8d49c98a41 Added simulation workload for blob granules and fixed some bugs 2021-08-26 13:48:05 -05:00
Fuheng Zhao e328e2b5df format fix 2021-08-24 18:34:44 -07:00
Fuheng Zhao 5eb92ae65c reomve registry 2021-08-24 15:47:04 -07:00
Josh Slocum 5259af787d Switched blob implementation to use backup container 2021-08-24 13:47:47 -05:00
Fuheng Zhao 57652a0cae Format 2021-08-24 10:00:24 -07:00
Fuheng Zhao b65a66fab7 log redwood histogram seperatly 2021-08-24 09:57:39 -07:00
Lukas Joswiak d554205db3 Move common knob initialization macros to header 2021-08-23 11:41:01 -07:00
Lukas Joswiak 7998ec4232 Add atomicity to knobs 2021-08-23 11:41:00 -07:00
Neethu Haneesha Bingi 02b3ed3ff1 Adding deadline option to rocksdb calls. 2021-08-19 14:11:28 -07:00
Neethu Haneesha Bingi fbb393f998 Added readrange timeoout check and rocksdb read deadline option. 2021-08-19 14:11:28 -07:00
Neethu Haneesha Bingi 24ac173c95 Adding ReadRangeAction timeout, returning error and using timer_monotonic changes. 2021-08-19 14:11:28 -07:00
Neethu Haneesha Bingi 01e85610ab Cancelling the timedout reads with rocksdb storage. 2021-08-19 14:11:28 -07:00
Daniel Smith 45b40addb7
Merge pull request #5379 from neethuhaneesha/eagerReadsDisable
Disabling option for removing eagerReads for ClearRange mutations.
2021-08-19 12:39:30 -04:00