b3698ac921
Merge pull request #3060 from xumengpanda/mengxu/fr-agent-fix-PR
...
Fix fastrestore_agent errors when run in non-simulation mode
2020-04-30 22:46:39 -07:00
fb1c456a2d
FastRestore:Change default knob value
2020-04-30 19:15:32 -07:00
7cebe743f9
A number of bug fixes of rare correctness errors
2020-04-29 13:50:13 -07:00
76fb345dd1
Merge branch 'master' into feature-tree-broadcast
...
# Conflicts:
# fdbrpc/FailureMonitor.actor.cpp
2020-04-29 09:51:22 -07:00
8dd05405eb
FastRestore:Guard knob with BUGGIFY
...
Prevent knob to be randomly set.
2020-04-28 16:05:40 -07:00
b7f5f3be48
merge in master
2020-04-28 13:11:47 -07:00
ee5051792a
Add BUGGIFY back to DESIRED_TEAMS_PER_SERVER knob
...
Kudos to A.J. who found this bug.
2020-04-24 12:54:31 -07:00
c87aa33941
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/go/src/fdb/generated.go
# documentation/sphinx/source/api-common.rst.inc
# documentation/sphinx/source/api-ruby.rst
# documentation/sphinx/source/release-notes.rst
# fdbclient/FailureMonitorClient.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/vexillographer/fdb.options
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-04-23 13:47:53 -07:00
0c84ad4bc6
Merge pull request #2917 from bnamasivayam/fail-slow-ss
...
Mark the storage servers that are continually lagging as unhealthy
2020-04-22 23:18:35 -07:00
d0cc2a1ee4
added logging for parallel peeks on TLogs
2020-04-22 14:24:45 -07:00
ba3e2af473
Merge commit '5288033bcfe40c3ade97c8bf2d04cf31b3f16cb1' into feature-tree-broadcast
2020-04-17 15:17:37 -07:00
4a05910c6c
Buggify FASTRESTORE_GET_RANGE_VERSIONS_EXPENSIVE knob for true and false
2020-04-17 11:20:57 -07:00
2d9e9a0502
FastRestore:Use knob to guard the expensive way to get range versions
2020-04-17 10:02:58 -07:00
dd951afe92
Merge pull request #2735 from ajbeamon/grv-proxy-perf-improvements
...
Improve proxy GRV latencies and responsiveness
2020-04-14 16:45:48 -07:00
08c8fac2af
Merge pull request #2943 from jzhou77/backup-fix
...
Buggify master recruit backup worker delay
2020-04-14 16:20:28 -07:00
9d6f2352d9
Merge commit 'cf01233f28a2c42908656a39f458a4475c1d44a3' into grv-proxy-perf-improvements
...
# Conflicts:
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/MasterProxyServer.actor.cpp
2020-04-14 13:49:09 -07:00
dbc9c23193
FastRestore:Loader:Send mutations at different versions in the same message to appliers
...
This increases the bandwidth sent from loaders to appliers.
2020-04-12 10:46:58 -07:00
87b8aae3e5
Buggify master recruit backup worker delay
2020-04-11 22:37:04 -07:00
8f78912483
knobified parameter
2020-04-11 20:54:17 -07:00
c0323f8a28
Merge commit '77f01193cbdf2ac5766131ffc8c71e8c88a0ed6b' into feature-tree-broadcast
2020-04-10 17:05:18 -07:00
07cc0a8d74
code cleanup
2020-04-10 17:02:11 -07:00
fe4bf3092f
Buggify DESIRED_TEAMS_PER_SERVER between 1 and 10
2020-04-10 09:43:01 -07:00
131ceeae1c
Adjust knobs.
2020-04-06 23:45:49 -07:00
73272fc72e
Version difference is now the diff between TLog versions and SS version.
2020-04-03 19:04:43 -07:00
884e92bb49
Atomically update dependent knobs
2020-04-01 15:18:49 -07:00
b1c3893d40
Fix some corner case bugs exposed by simulation.
...
In one case, when a SS joins the cluster and DD doesn't find any healthy server to form a team with the newly added server, then the SS does not get added to any team even when the other servers get healthy.
Another is an extreme case where a data center is down, and a SS in the active DC joins and then dies immediately but not before DD adds it to a destination team for a relocating shard which will result in DD waiting indefinitely for the dead data center to come back up for the cluster to be fully recovered.
2020-03-31 18:33:12 -07:00
40b17e1e9b
Remove a no longer unused knob
2020-03-26 13:04:00 -07:00
80d3fa1222
Add delay for master to recruit backup workers
...
This delay is to ensure old epoch's backup workers can save their progress in
the database. Otherwise, the new master could attempts to recruit backup
workers for the old epoch on version ranges that have already been popped. As
a result, the logs will lose data.
2020-03-20 20:15:08 -07:00
940bea102a
Add a knob to switch mutation logs for parallel restore
...
Knob FASTRESTORE_USE_PARTITIONED_LOGS, default is true to enable partitioned
mutation logs. Otherwise, old mutation logs are used.
2020-03-20 20:13:38 -07:00
34415f82b3
Merge pull request #2832 from xumengpanda/mengxu/backup-code-review-PR
...
Buggify upload delay when backup worker upload data to blob
2020-03-19 21:42:28 -07:00
94276076de
BackupWorker:Buggify upload delay
...
Add questions to code as well.
2020-03-18 19:04:45 -07:00
58a9bfa78b
Merge pull request #2820 from dongxinEric/fix/1977/add-back-trace-event-flush-failure-report
...
Fix/1977/add back trace event flush failure report
2020-03-18 16:11:44 -07:00
e08f0201f1
merge release 6.2 into master
2020-03-17 12:51:47 -07:00
ed4d02a3e4
Merge pull request #2812 from etschannen/feature-proxy-mem-limit
...
Limit the amount of requests the proxy can queue up in memory
2020-03-16 14:56:56 -07:00
d6d347f665
treat a tlog which takes a long time to create its disk queue as failed
2020-03-13 10:31:59 -07:00
243c268d9d
Limit the amount of requests the proxy can queue up in memory
2020-03-13 10:17:49 -07:00
5967ef5eab
Added back the changes that report trace log flush failures and fix the random crash
2020-03-12 14:34:19 -07:00
303df197cf
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/c/test/mako/mako.c
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/NativeAPI.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/LogRouter.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/fdbserver.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/flow.vcxproj
# flow/flow.vcxproj.filters
# versions.target
2020-03-06 18:22:46 -08:00
e219c1671f
Merge branch 'release-6.2' into feature-dd-region-queue
...
# Conflicts:
# fdbserver/Knobs.h
2020-03-04 16:25:38 -08:00
6d6f184e2f
added a knob which reverts the new queue behavior
2020-03-04 16:23:49 -08:00
b7834b2995
Merge pull request #2774 from etschannen/feature-dd-repopulate-priority
...
Make the DD priority of populating a region lower than machine failures
2020-03-04 16:15:18 -08:00
39610d15f8
Revert this change since it somehow introduced a random crash detected on circus
2020-03-04 16:14:38 -08:00
6296465e07
Make the DD priority associated with populating a remote region lower than machine failures
2020-03-04 14:07:32 -08:00
2c6f82e1ab
FastRestore:Add unit name to threshold knob name
2020-03-02 10:52:44 -08:00
1ef4cb432b
Merge branch 'master' into mengxu/fast-restore-robust-and-visibility-PR-v2
2020-03-01 20:08:07 -08:00
ad9b3fb4a8
DD:Add trace for detailed relocate shard info
2020-02-29 13:45:10 -08:00
5c562e3714
FastRestore:Optimize memory usage in updateHeartbeatTime
...
Only allocate memory once.
Reset vector element value for each loop.
Also ensure FASTRESTORE_HEARTBEAT_DELAY not 0
2020-02-28 20:52:25 -08:00
01c1a15caf
FastRestore:Applier:Limit fetch keys number in a txn in getAndComputeStagingKeys
2020-02-28 16:53:36 -08:00
62b9043ff6
FastRestore:DB can be destroyed before master unlock it in simulation
...
Because retore roles run as workload in simulation,
they do not know when DB is destroyed by the backup and restore test workload.
So if DB is destroyed earlier than restore master unlocks DB, which is rare,
restore master should abort the unlocking DB step.
2020-02-28 14:25:58 -08:00
993c6e478e
Merge branch 'master' into grv-proxy-perf-improvements
2020-02-28 14:25:42 -08:00
Jingyu Zhou