Commit Graph

282 Commits

Author SHA1 Message Date
Xiaoxi Wang cd13356964 solve review comments 2022-07-20 16:09:38 -07:00
Xiaoxi Wang 7865ead54d Merge branch 'main' into readaware 2022-07-19 13:35:33 -07:00
Markus Pilman 1de37afd52
Make TEST macros C++ only (#7558)
* proof of concept

* use code-probe instead of test

* code probe working on gcc

* code probe implemented

* renamed TestProbe to CodeProbe

* fixed refactoring typo

* support filtered output

* print probes at end of simulation

* fix missed probes print

* fix deduplication

* Fix refactoring issues

* revert bad refactor

* make sure file paths are relative

* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
Zhongxing Zhang 9c20b15f5a
Use enum variables to invoke Priority checking (#7514)
* Use enum variables to invoke Priority checking

* add an explicit isDataMovementForReadBalancing function
2022-07-14 09:06:56 -07:00
Xiaoxi Wang f5a81154ef move takeMoveKeysLock to DDTxnProcessor 2022-07-08 14:11:31 -07:00
He Liu bc5bfaffda
Shard based move (#6981)
* Shard based move.

* Clean up.

* Clear results on retry in getInitialDataDistribution.

* Remove assertion on SHARD_ENCODE_LOCATION_METADATA for compatibility.

* Resolved comments.

Co-authored-by: He Liu <heliu@apple.com>
2022-07-07 20:49:16 -07:00
Xiaoxi Wang ab2f2bc9b3 format code 2022-06-23 09:16:42 -07:00
Xiaoxi Wang c9ac30612f Merge branch 'main' of https://github.com/apple/foundationdb into readaware 2022-06-23 09:12:07 -07:00
Xiaoxi Wang ef0f415e3d add option; change to shared_ptr 2022-06-13 13:55:48 -07:00
Xiaoxi Wang 1de6c09307 use struct instead of tuple 2022-06-13 11:27:50 -07:00
Xiaoxi Wang c12a7a30ed
Update fdbserver/DataDistributionQueue.actor.cpp
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2022-06-13 08:22:48 -07:00
Xiaoxi Wang b99bd45730 format code 2022-06-09 12:36:20 -07:00
Xiaoxi Wang 178bab5faf Merge branch 'readaware-perf' into readaware 2022-06-09 12:23:41 -07:00
Xiaoxi Wang e5aa5fef22 merge upstream/main 2022-06-09 12:17:27 -07:00
Xiaoxi Wang 6ab12ea971 add storeTuple and unit test; refactor getSourceServersForRange 2022-06-09 12:16:12 -07:00
Bharadwaj V.R 24b2be1c4f Remove last-limited check from DDMountainChopper and DDValleyFiller 2022-05-30 21:57:34 -07:00
Xiaoxi Wang 9b8859f4f2 move keys out of StorageMetrics 2022-05-27 17:10:01 -07:00
Xiaoxi Wang 4fb2e90be9 add comment 2022-05-27 12:14:34 -07:00
Xiaoxi Wang 74748c20a0 a.fix heap-use-after-free caused by early noErrorsActors destroy 2022-05-27 10:14:10 -07:00
Xiaoxi Wang 85d1b59172 fix heap-use-after-free caused by early noErrorsActors destroy 2022-05-26 15:55:14 -07:00
Xiaoxi Wang 13a77dd5a2 change priority knob; change PromiseStream to FutureStream; remove comments; add on_sr check 2022-05-25 17:09:34 -07:00
Xiaoxi Wang 73624bcd2a Merge remote-tracking branch 'upstream/main' into readaware 2022-05-23 11:17:38 -07:00
Xiaoxi Wang 54e56af3db solve some review comments 2022-05-23 11:04:37 -07:00
Xiaoxi Wang 3fee0f33b5 change reference pointer 2022-05-23 00:12:48 -07:00
Xiaoxi Wang 27a60df36d fix rename bug 2022-05-22 23:37:32 -07:00
Xiaoxi Wang 0579b577ba remove polling interval; uncomment suppressFor 2022-05-22 23:35:39 -07:00
Xiaoxi Wang 4f28af1ab7 add comment 2022-05-19 10:50:24 -07:00
Xiaoxi Wang 8950822e36 add knobs; change knobs 2022-05-17 14:49:27 -07:00
Xiaoxi Wang f40a2a68fc add more statistics 2022-05-17 10:19:09 -07:00
Xiaoxi Wang 3b241955e7 add more informative trace info 2022-05-16 21:25:56 -07:00
Xiaoxi Wang dc5d2bff06 make ratekeeper don't limit read rebalance 2022-05-16 11:38:38 -07:00
Bharadwaj V.R 30922268d7 Minor readability improvement (I believe) to DD code; mostly, replacing .first, .second of pairs with better names through bindings) 2022-05-13 13:03:00 -07:00
Xiaoxi Wang e9e11bf53b change all criteria to knobs 2022-05-12 16:30:21 -07:00
Xiaoxi Wang 73e655a067 adjust (srcLoad - destLoad) / 10.0 2022-05-09 16:11:38 -07:00
Xiaoxi Wang b0a00effa0 set max shard bandwidth 2022-05-06 16:37:12 -07:00
Xiaoxi Wang 03607675e8 refactor GetTeamRequest 2022-05-04 17:42:49 -07:00
Xiaoxi Wang 964e0cecef format 2022-05-04 14:52:39 -07:00
Xiaoxi Wang 22eafcf7a2 rename trace event 2022-05-04 14:27:42 -07:00
Xiaoxi Wang ae66ed6c16 fix DataDistributionQueue time_out ; reset the rebalance poll time 2022-05-04 14:11:20 -07:00
Xiaoxi Wang a3d0b005dc reset several method use getShardMetrics 2022-05-04 00:00:03 -07:00
Xiaoxi Wang 7c37d172b9 solve some comments 2022-05-03 17:21:08 -07:00
Xiaoxi Wang 00b97ec829 add storage metric compare knob; timeThrottle with constant 2022-04-27 23:37:35 -07:00
Xiaoxi Wang 73c7240bfd code format 2022-04-26 17:34:34 -07:00
Xiaoxi Wang 9b211c48ee format code 2022-04-26 17:13:37 -07:00
Xiaoxi Wang 8cc6d9aaac determine timeThrottle and topK dynamically on the shard number 2022-04-25 16:59:20 -07:00
Xiaoxi Wang cc05f5e9db fix getMetrics keys bug 2022-04-24 21:21:18 -07:00
Xiaoxi Wang a0b45c2962 fix throttle bug 2022-04-24 15:51:18 -07:00
Xiaoxi Wang eefd0778c5 change canQueue to timeThrottle() 2022-04-22 15:26:44 -07:00
Xiaoxi Wang 4818f01de9 fix top10 shard index bug; add event detail; fix merge conflict 2022-04-22 14:14:58 -07:00
Xiaoxi Wang 0639810b66 merge upstream/main 2022-04-22 11:09:15 -07:00