73c7240bfd
code format
2022-04-26 17:34:34 -07:00
9b211c48ee
format code
2022-04-26 17:13:37 -07:00
8cc6d9aaac
determine timeThrottle and topK dynamically on the shard number
2022-04-25 16:59:20 -07:00
cc05f5e9db
fix getMetrics keys bug
2022-04-24 21:21:18 -07:00
a0b45c2962
fix throttle bug
2022-04-24 15:51:18 -07:00
eefd0778c5
change canQueue to timeThrottle()
2022-04-22 15:26:44 -07:00
4818f01de9
fix top10 shard index bug; add event detail; fix merge conflict
2022-04-22 14:14:58 -07:00
0639810b66
merge upstream/main
2022-04-22 11:09:15 -07:00
eb949ee3dc
change default shard size; enable valley filler best; top 10 random choice
2022-04-22 10:07:12 -07:00
04311d001e
topK shard random selection
2022-04-21 22:37:16 -07:00
960a13f233
format
2022-04-21 21:40:49 -07:00
f9e57396e2
completeDest
2022-04-21 21:23:42 -07:00
a8bfd0f481
move canQueue, 60s each source server, random select portion of shards/destServer
2022-04-20 23:12:34 -07:00
af9e5ba885
move canQueue, 60s each source server, random select portion of shards
2022-04-20 22:19:56 -07:00
131adec811
add canQueue
2022-04-20 15:28:03 -07:00
25e1e75d9e
enable destComplete
2022-04-20 13:32:04 -07:00
e0ae0942c4
fix inflight read division; temp destComplete fix; 0.1 constant poll time
2022-04-20 12:15:40 -07:00
a7a9de781e
update unittests
2022-04-16 22:51:55 -07:00
7b9432bdda
revert to original BgDDValleyFiller and BgDDMountainChopper for disk rebalance
2022-04-12 18:01:28 -07:00
8d3f851495
merge upstream/mainA
2022-04-12 17:03:09 -07:00
ed97a35dc0
Merge branch 'main' into readaware
2022-04-12 16:47:15 -07:00
5e96bacb5b
add new priority in RelocateData
2022-04-12 16:22:17 -07:00
61a1f7683b
fix dd command line read special key space error
2022-04-11 22:49:21 -07:00
6c841cd32b
merge readaware-better
2022-04-11 17:09:39 -07:00
129a7b5daf
Use boolean-param for GetTeamRequest params
2022-04-11 13:27:08 -07:00
411550580e
merge
2022-04-08 11:49:33 -07:00
7c83f63e46
fix substraction bug; test1,2,3 generate good result
2022-04-08 10:44:56 -07:00
e27b0d9ab5
Merge remote-tracking branch 'origin/main' into improve-snapshot-fault-tolerance
2022-04-07 23:30:16 -07:00
91930b8040
Remove getMinReplicasRemaining PromiseStream.
...
Instead, in order to enforce the maximum fault tolerance for snapshots,
update getStorageWorkers to return the number of unavailable storage
servers (instead of throwing an error when unavailable storage servers
exist).
2022-04-07 23:23:23 -07:00
c858b0eaa9
update penalty, reset sim skip to 3; add read_underutil to relocator priority
2022-04-07 11:59:40 -07:00
6fa60a2cf9
fix substraction typo
2022-04-06 23:03:25 -07:00
150b4318aa
refactor datadistribution command; try dual-mode code
2022-04-06 22:10:23 -07:00
aba9d85560
merge main
2022-04-06 09:57:52 -07:00
b6594aebbf
change inflight substraction priority (not non-negligible performance influence
2022-04-04 11:34:41 -07:00
70f378bacc
Restrict write access to getUnhealthyRelocationCount
2022-04-03 23:47:54 -07:00
253db642be
Add MAX_SNAPSHOT_FAULT_TOLERANCE knob
2022-04-03 22:31:45 -07:00
9e06881673
fix destination limiting and cancelling logic in move_to_removed_server case
2022-03-31 14:05:15 -05:00
20a0b8d761
enable sim skip; add readInFlight methods and inflight Penalty; add delayed inflight substraction:
2022-03-31 09:57:00 -07:00
b15f0eb2b0
add ReadInFlight
2022-03-28 14:20:07 -07:00
25f757ba28
fix merge conflict bug caused by removal of cancellable(true)
2022-03-28 14:13:44 -07:00
9799b3dc99
change teamSorter details
2022-03-24 22:20:58 -07:00
d93b57dd88
conflict solving
2022-03-24 20:45:51 -07:00
1b631a9263
solve conflict with main
2022-03-24 16:29:11 -07:00
cc934ff1d7
Merge pull request #6632 from sfc-gh-jslocum/dd_better_cancelling
...
Better cancelling logic that reflects whether move has actually started
2022-03-24 16:28:01 -07:00
b811a62b65
fix teamSorter usage bug
2022-03-24 13:16:10 -07:00
746778fa0a
CPU reading
2022-03-23 11:18:58 -07:00
0186e00256
set team too similar criteria for read rebalance
2022-03-22 12:01:42 -07:00
6390d93efd
Merge pull request #6646 from sfc-gh-tclinkenbeard/fix-copyright-headers
...
Update copyright header dates
2022-03-21 16:49:20 -07:00
a71099471b
Update copyright header dates
2022-03-21 13:36:23 -07:00
305dead7f5
Merge remote-tracking branch 'origin/main' into change-rebalance-teams-signature
2022-03-18 21:12:47 -07:00
4b92f8f546
add relocate reason and set teamSorter in relocator
2022-03-18 16:39:31 -07:00
4a0720e8ad
Provide destTeam parameter to rebalanceTeams again.
...
There is a wait before the load bytes of the dest team is used, so the
last commit inadvertently caused a behaviour change. Instead, update the
comment for the function, and pass const IDataDistributionTeam
references.
2022-03-18 10:25:13 -07:00
5fbdbb5c4a
Better cancelling logic that reflects whether move has actually started
2022-03-18 10:43:24 -05:00
a04ca65ad5
Change signature of rebalanceTeams
2022-03-17 23:27:33 -07:00
67eba5ec7c
Limiting DD Moves by destination SS.
2022-03-15 13:52:19 -05:00
10eb082888
format code; fix DD ignore option bug"
2022-03-15 10:31:43 -07:00
d38b827906
fix DD ignore bug
2022-03-13 23:47:07 -07:00
8d20ee8432
change storage metrics of read sample calculation
2022-03-07 17:36:43 -08:00
970a84bf63
fix teamSorter setting
2022-03-03 12:58:45 -08:00
a28b0f4361
Merge branch 'main' of https://github.com/apple/foundationdb into readaware
2022-03-03 11:15:18 -08:00
a05f6f851a
fix reference assign bug
2022-03-02 21:56:03 -08:00
f48ba989c3
add status
2022-03-01 15:27:27 -08:00
5e74b5006e
fix uninitialized member
2022-02-28 10:22:32 -08:00
40a1f562a7
temporary change special key for data distributor
2022-02-25 11:01:23 -08:00
a07e413eae
metrics comparator; rebalanceReadLoad()
2022-02-24 16:41:01 -08:00
250a88e682
Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement.
2022-02-24 12:25:52 -08:00
1afef44f19
fix: self->shardsAffectedByTeamFailure->moveShard must be called without any waits after getting the destination team or we could miss failure notifications for the storage servers in the destination team
2021-11-23 15:26:38 -08:00
d1959122af
consider wiggling when waitUntilHealthy
2021-10-27 09:08:37 -07:00
8761960cdc
fix roll trace event issue for data distribution(master)
...
Description
Testing
2021-09-24 15:40:17 -07:00
814e3c729f
fix roll trace event issue for data distribution(master)
...
Description
Testing
2021-09-24 15:40:17 -07:00
c10dd0df4b
fix roll trace event issue for data distribution
...
Description
Testing
2021-09-24 15:40:17 -07:00
abf73047ca
Enforce std:: specifier rather than using namespace
2021-09-16 19:40:28 -07:00
3442ebd3b7
Fix more -Wreorder-ctor warnings across many files
2021-07-24 11:20:51 -07:00
8efe3b296a
Delete remaining extern declarations for noUnseed
2021-07-08 19:19:22 -04:00
0493d149e6
wait remove
2021-06-21 05:18:42 +00:00
d46fccc30f
Revert "Revert "Properly set simulation test for perpetual storage wiggle and bug fixing""
...
This reverts commit ad576e8c20
2021-06-11 22:58:05 +00:00
ad576e8c20
Revert "Properly set simulation test for perpetual storage wiggle and bug fixing"
2021-06-11 09:07:45 -07:00
17ac91bac4
Merge pull request #4929 from sfc-gh-xwang/ppwtest
...
Properly set simulation test for perpetual storage wiggle and bug fixing
2021-06-10 14:09:50 -07:00
4220a548ce
use the same health check as exclude to avoid 'best team get stuck'
2021-06-09 22:51:46 +00:00
371a38e6e5
Merge remote-tracking branch 'origin/master' into remove-extra-copies
2021-06-07 10:26:06 -07:00
85cd2b9945
add perpetualStorageWiggler
2021-05-20 23:31:08 +00:00
f28ac955c3
Remove unnecessary temporary objects while growing objects of type std::vector<std::pair<A, B>>
2021-05-10 16:32:50 -07:00
5c2d7b6080
Create RangeResult type alias
2021-05-03 13:14:16 -07:00
df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
4ee97c0784
Use clang-tidy to automatically fix missing overrides
...
Use `clang-tidy -p . $file -checks='-*,modernize-use-override' -header-filter='.*' -fix`
to fix missing overrides, and then use git clang-format to reformat just
those changes. This went pretty well for most files.
Formatting the following files went off the rails, so I'm going to
follow up with a commit that's just clang-tidy and no clang-format.
- fdbclient/DatabaseBackupAgent.actor.cpp
- fdbclient/FileBackupAgent.actor.cpp
- fdbserver/OldTLogServer_4_6.actor.cpp
- fdbmonitor/SimpleIni.h
- fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
2021-01-26 02:04:12 +00:00
7fbc4d7391
Resolve conflicts
2020-12-04 23:58:42 +00:00
877997632d
Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
...
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
dc2bac5670
Resolve conflicts
2020-11-24 19:09:42 +00:00
1f541f02be
Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge
...
Merge, leaving conflict markers for now
2020-11-24 16:55:34 +00:00
d64cf8b9e3
Merge branch 6.3 into master
2020-11-17 11:22:45 -08:00
489ba20641
Fix several merge issues
2020-11-16 14:46:36 -08:00
d0391db862
Merge branch 'release-6.2' into release-6.3
2020-11-16 10:15:23 -08:00
15d10d68cd
Cleanup DataDistributionQueue.actor.cpp and storageserver.actor.cpp
2020-11-09 15:23:51 -08:00
2909e35415
When moving dispaching shard relocations, log the source and destination storage servers.
2020-11-05 16:13:18 -08:00
cedcfc7d87
Removed unused uid field inside ParallelTCInfo
2020-10-26 16:48:09 -07:00
9ef29d0cea
Changed getTeamID() to return a string instead of UID as suggested by reviews.
2020-10-26 16:44:52 -07:00
7ebb2e5c09
Piggy back this PR to polish more TraceEvent by:
...
- Making it clear that it's tracking machine team info or server team info
- Added ID to both machine team and server team for better trackability
- Attach distributor id to some trace events.
2020-10-26 16:44:09 -07:00
e0b1f95740
Merge remote-tracking branch 'origin/master' into remove-global-ddenabled-flag
2020-10-21 18:22:08 -07:00
8cc3e4d3c6
Merge release-6.3 into master
2020-10-19 22:51:56 -07:00
652d753daf
Remove global ddEnabled flag
2020-10-17 11:23:52 -07:00
Xiaoxi Wang