e94de126de
merge with master
2020-08-14 01:59:09 +00:00
8f1deaa47e
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-12 23:32:23 +00:00
b17667323e
Resolve review comments
2020-08-12 15:15:25 -07:00
3ac6996844
Increased some knobs to throttle the spammy read hot logging. Also added more details inside the read hot log to make it useful
2020-08-12 14:27:34 -07:00
7ee6ca342e
merge with master
2020-08-11 01:01:15 +00:00
cae6f04070
sample on cost instead of bytes
2020-08-10 22:30:06 +00:00
87bde08695
FR:fix:sending samples may have broken promise error
...
The error should not skip the ack reply from loader to controller, saying
the file has been loaded.
Otherwise, restore controller can get stuck at waiting for the ack reply.
2020-08-06 10:41:18 -07:00
fe5902994c
Merge pull request #3605 from apple/release-6.3
...
Merge Release 6.3 to master
2020-08-05 23:37:44 -07:00
7992cef025
FR:Fix sample network pkg can be too big
2020-08-04 22:35:21 -07:00
d1cc87452c
merge with master; solve conflicts; solve initialization;
2020-08-02 22:44:07 +00:00
0352e8ee0b
pick busiest commit tag periodically
2020-08-02 18:38:56 +00:00
2f5293fcc7
Introduce knob FASTRESTORE_USE_LOG_FILE and FASTRESTORE_USE_RANGE_FILE
2020-07-31 10:40:29 -07:00
8cace30bb2
FastRestore:change TXN_BATCH_MAX_BYTES default to 1KB from 1MB
2020-07-30 16:43:35 -07:00
ad915e462e
Add knob FASTRESTORE_NOT_WRITE_DB to skip writting to DB
2020-07-30 10:17:17 -07:00
a49cb41de7
Merge branch 'release-6.3'
...
# Conflicts:
# CMakeLists.txt
# cmake/ConfigureCompiler.cmake
# fdbserver/Knobs.cpp
# fdbserver/StorageCache.actor.cpp
# fdbserver/storageserver.actor.cpp
# flow/ThreadHelper.actor.h
# flow/serialize.h
# tests/CMakeLists.txt
2020-07-29 00:31:55 -07:00
c3348be3a0
changed knobs to improve master recovery speeds
2020-07-27 10:59:06 -07:00
937df4f839
Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-lifetimetoken-fix
...
# Conflicts:
# documentation/sphinx/source/release-notes/release-notes-630.rst
2020-07-27 10:03:02 -07:00
2ac7df8a18
Merge pull request #3563 from xumengpanda/mengxu/fr-filesize-PR
...
FastRestore:Add trace for file size and bc progress
2020-07-25 20:08:57 -07:00
99d8399f4e
FastRestore:Add trace for file size and bc progress
2020-07-25 19:12:25 -07:00
be67e9cfc7
wait for the correct cluster controller interface before starting master recovery
2020-07-20 11:29:37 -07:00
e646361501
Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-redwood
2020-07-20 07:25:29 -07:00
d05b7ee785
Pager remap remover now accumulates a configurable version interval of page updates behind the oldest retained version which are used to coalesce updates to the same original page ID to reduce write amplification for many workloads.
2020-07-20 04:08:33 -07:00
ef8c1060a2
Merge branch 'master' into mengxu/tmp-merge-6.3
2020-07-13 10:15:56 -07:00
b09dddc07e
Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
...
# Conflicts:
# cmake/ConfigureCompiler.cmake
# documentation/sphinx/source/downloads.rst
# fdbrpc/FlowTransport.actor.cpp
# fdbrpc/fdbrpc.vcxproj
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/LogSystemPeekCursor.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/Status.actor.cpp
# fdbserver/storageserver.actor.cpp
# flow/flow.vcxproj
2020-07-10 15:06:34 -07:00
4492a50e72
Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-ha-fixes
...
# Conflicts:
# documentation/sphinx/source/mr-status-json-schemas.rst.inc
# documentation/sphinx/source/mr-status.rst
# fdbclient/Schemas.cpp
2020-07-10 10:37:43 -07:00
28b6281848
Increase latency sample size by a factor of 10.
2020-07-10 08:52:59 -07:00
dd10dbe7c7
Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-ha-fixes
2020-07-09 23:09:14 -07:00
717242a0ee
reset WAN network connections every 5 minutes is responses take more than 500ms
2020-07-09 22:50:47 -07:00
04d1217941
Track statistics about server-side request latency on each process, to include min, max, mean, and various percentiles.
2020-07-09 16:39:15 -07:00
0e2f5e8bb5
Added a flow lock to prevent too many source server fetches from happening at the same time and running the data distributor out of memory
2020-07-09 10:38:19 -07:00
7546ed3edf
Remove trace events
2020-07-02 15:17:11 -07:00
ed89d69916
Change order between reporting commit version and setting committedVersion
2020-07-02 15:13:28 -07:00
ab95de6e1e
Add a warning message
2020-06-30 17:43:07 -07:00
e613d85895
Fix errors in joshua related to PR_3307
2020-06-30 17:22:59 -07:00
f3302833ce
Merge pull request #3435 from apple/release-6.3
...
Merge Release 6.3 to master
2020-06-30 10:08:28 -07:00
efb61bcac0
Rename knob to FASTRESTORE_TXN_EXTRA_DELAY
2020-06-29 21:16:30 -07:00
4b35685e99
FastRestore:Adjust knob based on circus test
2020-06-29 10:13:18 -07:00
78c45c1200
Knob for txn delay and add back FlowLock to control txn concurrency
2020-06-27 10:13:34 -07:00
f211a54593
Merged from upstream master
2020-06-13 16:47:12 -07:00
739767b838
Delay cluster controller candidacy for all worst fit processes, not just storage servers.
2020-06-10 09:59:56 -07:00
c3c7db6a9d
Added flowlock around reads done on the write path so that those reads cannot starve reads to support reads done by the storage server.
2020-06-09 17:00:21 -07:00
a038a02cdd
Serve GetReadVersion through master
2020-06-09 11:16:23 -07:00
d128252e90
Merge release-6.3 into master
2020-05-22 09:25:32 -07:00
c0c15130b8
Merge pull request #3172 from jzhou77/backup-fix
...
Limit memory usage of backup workers
2020-05-20 15:46:11 -07:00
4a827c9304
Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-redwood
2020-05-19 03:08:26 -07:00
e8626724f9
More refactor of Redwood metrics. Added per-level BTree stats. Added Pager total physical disk reads/writes, and uncacheable read hit/miss counts. Added logging to TraceEvents.
2020-05-16 02:51:57 -07:00
89ae1200dd
Increase backup worker memory buffer size in simulation
...
Lower limit causes backup worker to become stuck.
2020-05-15 09:45:52 -07:00
32f4639168
Refactored page remap removal. The process is now called remap cleanup, does batches of remap entries in parallel, coalesces remaps of the same page ID to skip unnecessary writes, and has some knobs for controlling it. FIFOQueue now has peek() to support remap cleanup version lag limits. Added counters for remap cleanup and lazy subtree deletion. Refactored Redwood counters, normalized and grouped their names.
2020-05-15 02:10:51 -07:00
17915e13b0
Limit memory usage of backup workers
2020-05-14 13:24:56 -07:00
421a6581c1
Lazy delete cycles now run more often and not just when the btree is having mutations actively merged into it. Specifically, it is launched upon recovery, then stopped and relaunched after every commit. This will make better use of I/O in between calls to commit(). Also added knobs to control lazy delete cycle parallelism and work limits.
2020-05-13 02:27:03 -07:00
Xiaoxi Wang