3f9d9d8b84
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# cmake/FlowCommands.cmake
# documentation/sphinx/source/release-notes.rst
# fdbclient/StorageServerInterface.h
# fdbserver/DataDistributionTracker.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# flow/Knobs.h
# flow/Platform.cpp
# versions.target
2020-01-16 18:37:47 -08:00
9b80498180
Added a trace event to warn if a shard is merged before enough time has elapses from becoming low bandwidth
2020-01-10 14:58:38 -08:00
c2608f0af9
fix: completeSources could be larger than the teamSize, so we need to check all completeSources
...
we do not need to track bestSize, since all teams in the list will be the same size
2020-01-10 14:46:40 -08:00
ab7071932f
Data distribution no longer attempts to pick teams which share members of the source unless the team matches exactly
2020-01-09 16:59:37 -08:00
59738e8ef1
fixed compiler error
2019-11-22 16:19:34 -08:00
3c769fcf60
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# fdbserver/ClusterController.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# versions.target
2019-11-22 15:39:19 -08:00
3a3ab5664b
fix: team trackers for bad teams that contain a removed servers must be cancelled or the cluster will falsely report those teams as failed
2019-11-22 10:20:13 -08:00
d4de608bb6
Fix OPEN_FOR_IDE build
2019-10-25 10:42:22 -07:00
f8e44d2f71
fix: If a storage server was offline, it would not be checked for being in an undesired dc
2019-10-23 23:04:39 -07:00
d2b6626d5c
Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed
2019-10-21 13:47:06 -07:00
688940b685
merge 6.2 into master
2019-10-21 11:43:46 -07:00
b1fd6b4443
addressed review comments
2019-10-18 09:43:25 -07:00
896701006f
addressed code review changes
2019-10-16 11:30:20 -07:00
86bcb84b45
Raised the data distribution priority of splitting shards above restoring fault tolerance to avoid hot write shards
2019-10-11 17:50:43 -07:00
34baa37e60
Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed
2019-10-10 10:14:58 -07:00
1bd6151f54
Update fdbserver/DataDistribution.actor.cpp
...
Co-Authored-By: Jingyu Zhou <jingyuzhou@gmail.com>
2019-10-09 21:17:03 -07:00
26e1d565f6
StorageServerTracker:Fix OOM bug caused by server healthyness toggles infinitely
...
When there is only one healthy team, the bug will set a server's status as unhealthy;
which causes the healthyTeam to 0, triggering StorageServerTracker to loop back;
which resets the server's status to healthy, and thus the healthyTeam to non-zero.
This pattern will cause infinite loop.
Infinite loop will prevent TraceEvent from flushing, which causes
TraceEvent to use most of memory and out-of-memory.
Kudos to JingYu Zhou (jingyu_zhou@apple.com ) who is the main contributor who found the bug!
2019-10-09 17:45:09 -07:00
eb41e32876
add extra dd safety check to deny exclude if only 1 team exists
2019-10-08 16:10:09 -07:00
ff563c1672
fixed trace line description
2019-10-07 12:45:27 -07:00
d146f1d636
Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed
2019-10-07 11:27:15 -07:00
22c3fa867c
Merge pull request #2074 from xumengpanda/mengxu/fix-correctness-bug
...
DD:fix:getTeam may fail to get a team when it can
2019-10-07 09:33:57 -07:00
cc41cd610d
force recheck in tracker if team contains failed server
2019-10-04 14:02:12 -07:00
6dae95a4ca
Update fdbserver/DataDistribution.actor.cpp
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-10-04 10:02:11 -07:00
31332da57d
move option setting inside retry loop
2019-10-03 13:24:28 -07:00
abb46d527b
fixed implementation of watches in TrackExcludedServers
2019-10-03 13:18:05 -07:00
d96a7b2c69
Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed
2019-10-03 09:47:45 -07:00
5286523a43
StorageServerTracker:Distinguish wrongDC from invalidLocality
2019-10-02 14:50:53 -07:00
9e7dfa358c
Replace isCorrectDC with isCorrectLocality
...
This incorporates the change for defending DD from misconfigured
locality entries.
The check for misconfigured locality was in keyValueStoreTypeTracker,
but the storage engine switch PR moves the isCorrectDC checking out of
the tracker and move it into storageServerTracker
2019-10-02 14:05:43 -07:00
d0147e5e5d
Merge branch 'release-6.2' into mengxu/merge-release620-to-master-v3
...
Resolved Conflicts:
documentation/sphinx/source/release-notes.rst
fdbserver/DataDistribution.actor.cpp
versions.target
2019-10-02 13:22:56 -07:00
4b5080fbea
added a few more missing data distribution priorities
2019-09-27 19:39:53 -07:00
324d0bd3b0
Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-cleanup-mutations
2019-09-27 19:15:14 -07:00
3bb62e008c
lowered the priority of some delays in data distribution so that the process will prefer other work
2019-09-27 18:33:13 -07:00
efa2f5df6b
avoid double overwrite on failed servers and reordered removal of keys
2019-09-26 13:52:53 -07:00
450a09e117
Code Review Changes
2019-09-24 15:48:50 -07:00
32ebd08f9f
DD:Trigger storage recruitment when an invalid address locality is corrected
2019-09-24 13:35:38 -07:00
515689d07b
Update fdbserver/DataDistribution.actor.cpp
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-09-18 14:45:18 -07:00
d175b61a62
DD:Trace when invalid locality is corrected
...
Change getWorkers param from txn handler to db
2019-09-18 13:52:54 -07:00
40ad6f0931
created new reply types to work with flatbuffer serialization requirements
2019-09-18 13:40:18 -07:00
471e283128
Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed
2019-09-18 11:49:07 -07:00
93bbc26a35
set:erase:Use return value of erase iterator as next iterator
2019-09-17 15:28:30 -07:00
d2fd1f4931
DD:MisconfiguredLocality:Fix review comments
2019-09-17 13:04:21 -07:00
37d2318eed
DD:Handle worker with incorrect locality
...
When a worker has incorrect locality, the worker will be excluded from
storage recruitment.
When the worker has its locality corrected by system operators,
the worker will be reincluded for storage recruitment.
2019-09-14 12:12:56 -07:00
c3960aba17
DD:initializeStorage:Exclude worker with invalid locality
2019-09-13 22:05:41 -07:00
75460089e1
DD_VALIDATE_LOCALITY:Add comment for our future selves
...
When we add simulation test that misconfigure a cluster by not setting some
locality entries, we should set DD_VALIDATE_LOCALITY always true.
Otherwise, simulation tests may fail.
2019-09-13 16:26:54 -07:00
78b8e48cef
DD:ValidLocality:Resolve review comment
2019-09-13 15:35:16 -07:00
e1dcdbf3d2
LocalityData:Remove verbose check for valid locality
2019-09-13 15:11:13 -07:00
8970d9858b
DD:isValidLocality:A generic way to check any replicationPolicy
2019-09-13 14:55:51 -07:00
1196841b3d
DD:IsValidLocality:Clang format
2019-09-13 13:56:43 -07:00
e8878b16d4
DD:Valid locality includes an empty but set locality entry
2019-09-13 13:55:46 -07:00
1596e2e4a5
DD:TCMachine:Use processID as machineID if zoneID is unset
2019-09-13 13:43:41 -07:00
Evan Tschannen