Commit Graph

172 Commits

Author SHA1 Message Date
RenxuanW 551dfa6ad8 Move description of RESTORE_IGNORE_LOG_FILES to header file. 2021-04-07 10:21:24 -07:00
RenxuanW fadc9cccee Use knob RESTORE_IGNORE_LOG_FILES in restore.
Rename IGNORE_LOG_FILES to RESTORE_IGNORE_LOG_FILES. Also, this knob should be used in regular restore, not parallel restore.
2021-04-07 10:05:56 -07:00
RenxuanW 1b6ad42db8 Use a knob to completely ignore log files 2021-04-06 19:07:01 -07:00
Nim Wijetunga 73571ad52e Merge branch 'master' of github.com:sfc-gh-nwijetunga/foundationdb into network_busyness
* 'master' of github.com:sfc-gh-nwijetunga/foundationdb: (55 commits)
  Port lost changes from #4004
  Fix markdown
  Factor our refreshTransaction gadget
  Align FastAllocator memory to 4096 for size >= 4096
  Fix heap-use-after-free's
  Disable lsan in joshua
  Manually apply changes made to BackupContainer.actor.cpp on release-6.3 to the new files where that code is located.
  add release branch instructions
  Fix PR number in release note link
  fixed a simulation bug where a process on an unreliable machine would be considered reliable by the simulator
  Add tini as PID 1 to docker image (#4363)
  Fix typo
  Fix broken_promise bug
  Explain FastAllocator<4096> special case
  Inform LSAN that we have unaligned pointers
  apply clang-format to flow/Platform.h
  Fix typos
  Address review comments
  Change the macro that forbids exit() calls to be a static assertion
  Add versionstamp serialization information
  ...
2021-03-18 18:58:43 +00:00
A.J. Beamon 25c4880ebe Merge branch 'release-6.3' into merge-release-6.3-into-master (temporarily discard all changes to BackupContainer.actor.cpp)
# Conflicts:
#	fdbclient/BackupContainer.actor.cpp
#	fdbserver/Knobs.h
2021-03-15 16:41:04 -07:00
Nim Wijetunga f0d79b3d86 Inital implementation of network busyness 2021-03-15 23:23:56 +00:00
Markus Pilman cfbaaa6001 Merge remote-tracking branch 'origin/release-6.2' into release-6.3 2021-03-10 14:30:47 -07:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Steve Atherton 318b862aa3 Apply clang-format to backup changes. 2021-03-08 01:49:29 -08:00
Steve Atherton a397d8625f AsyncFileNonDurable now has a flag for whether or not to emulate AIO mode, so the previous behavior limits are preserved when this flag is set. AsyncFileDurable writes which are selected to be durable are again done in a single write, and the logic to handle non-aligned writes for non-AIO mode is more clear and commented. In simulation, backup files on local filesystems are written with random buffer sizes. Backup container unit test now uses more random file sizes and limits memory consumption for all files being written in parallel. 2021-03-07 22:21:10 -08:00
Steve Atherton 1d00c41385 Bug fixes: Avoid empty writes, avoid 0 length truncation, and make AsyncFileNonDurable support writes at arbitrary offsets and lengths. Increase randomness of IBackupFile appends in backup container unit test. 2021-03-05 17:45:36 -08:00
FDB Formatster 8a8c488ede apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-05 18:13:38 -06:00
sfc-gh-tclinkenbeard 7558cd8e06 Merge remote-tracking branch 'origin/master' into simplify-global-knobs 2021-01-26 10:00:25 -08:00
Andrew Noyes ff7d306b09 Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
Include conflict markers for now. Will resolve.
2021-01-15 18:04:09 +00:00
Andrew Noyes 6d6b2843da Merge remote-tracking branch 'upstream/release-6.2' into anoyes/merge-release-6.2
I'm including the conflict markers for now and I'll resolve them in a
subsequent commit
2021-01-14 17:20:11 +00:00
sfc-gh-tclinkenbeard e29ed3bf99 Remove createGlobal*Knobs functions 2021-01-13 12:14:04 -08:00
Steve Atherton a860778b51 Add write buffering to BackupContainerLocalDirectory::BackupFile to greatly reduce the number of IAsyncFile::write() calls made when writing backup data. The buffer size is controlled by a knob. 2021-01-09 07:57:48 -08:00
sfc-gh-tclinkenbeard b46384e1b8 Initialize client and flow knobs 2020-11-22 09:56:18 -08:00
sfc-gh-tclinkenbeard b1e3478267 Simplified global knobs 2020-11-21 13:27:48 -08:00
Markus Pilman 268ba0bddc Merge remote-tracking branch 'origin/release-6.3' into merge-6.3 2020-10-01 14:14:06 -06:00
sfc-gh-tclinkenbeard e2705668be Lower WATCH_TIMEOUT default value to 30.0 2020-09-30 11:52:51 -07:00
Young Liu 3278e21b0b Change description of GRV proxy and Commit proxy in docs and add backward compatibility support for configuring proxies in fdbcli 2020-09-25 10:50:48 -07:00
Young Liu cc5bc16bd8 Rename more places from proxy to commit proxy 2020-09-15 22:29:49 -07:00
Young Liu 35bef73a1c Rename proxy to commit proxy 2020-09-10 17:44:15 -07:00
Young Liu 87693cae81 merge master branch and resolve conflicts 2020-09-02 13:44:33 -07:00
Evan Tschannen 12edadd059 Merge branch 'release-6.3'
# Conflicts:
#	CMakeLists.txt
#	fdbclient/Knobs.cpp
#	fdbclient/MasterProxyInterface.h
#	fdbrpc/simulator.h
#	fdbserver/MasterProxyServer.actor.cpp
#	tests/fast/CycleAndLock.txt
#	tests/fast/TxnStateStoreCycleTest.txt
#	tests/fast/VersionStamp.txt
#	tests/slow/ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt
#	tests/slow/ParallelRestoreOldBackupCorrectnessCycle.txt
#	versions.target
2020-08-31 19:33:34 -07:00
Evan Tschannen 808f8309b1 disable tag_encode_key_servers because it does not help recovery times (the CPU on the proxies decoding the tags is more than the saving for the smaller size) 2020-08-31 09:29:46 -07:00
Young Liu e87327b33b Merge master branch and keep master proxy reporting txn cost estimation to ratekeeper 2020-08-29 12:47:35 -07:00
Xiaoxi Wang 24dbb3961c Knob adjustment 2020-08-17 22:38:22 +00:00
Xiaoxi Wang f3ecf14601 change midShardSize type and other details 2020-08-12 17:49:12 +00:00
Young Liu 16ef2bd3bd Pending commit 2020-08-12 10:34:07 -07:00
Young Liu 79ce16650d merge master branch 2020-08-11 19:22:10 -07:00
Xiaoxi Wang cae6f04070 sample on cost instead of bytes 2020-08-10 22:30:06 +00:00
Xiaoxi Wang 696e77c94e query midShardSize from proxy 2020-08-10 20:13:44 +00:00
Young Liu d6a23a4d6b Resolve comments to make GRV proxy a separate process class 2020-08-06 00:01:57 -07:00
Xiaoxi Wang 13307679c5 use median shard size" 2020-08-05 03:57:25 +00:00
Xiaoxi Wang 48094ea17e add INCOMPLETE_SHARD_PLUS knob 2020-08-04 03:18:05 +00:00
Xiaoxi Wang 52073738c0 fix bugs; change some KNOB values 2020-08-03 22:18:34 +00:00
Xiaoxi Wang d1cc87452c merge with master; solve conflicts; solve initialization; 2020-08-02 22:44:07 +00:00
Xiaoxi Wang 0352e8ee0b pick busiest commit tag periodically 2020-08-02 18:38:56 +00:00
Xiaoxi Wang 92c1112c74 consider clear single key 2020-08-01 18:20:13 +00:00
Xiaoxi Wang c3a629588f add client transaction tag sample 2020-07-31 19:08:42 +00:00
Young Liu f7b76a92af pass joshua 2020-07-29 07:26:55 -07:00
Evan Tschannen a49cb41de7 Merge branch 'release-6.3'
# Conflicts:
#	CMakeLists.txt
#	cmake/ConfigureCompiler.cmake
#	fdbserver/Knobs.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	flow/ThreadHelper.actor.h
#	flow/serialize.h
#	tests/CMakeLists.txt
2020-07-29 00:31:55 -07:00
Evan Tschannen c3348be3a0 changed knobs to improve master recovery speeds 2020-07-27 10:59:06 -07:00
Young Liu 525f10e30c Merge master branch 2020-07-22 16:08:49 -07:00
Young Liu 302cf5c45f Remove debug trace events 2020-07-22 12:20:22 -07:00
Meng Xu b2a3b4fd83 Merge branch 'master' into mengxu/merge-6.3-PR 2020-07-20 11:34:18 -07:00
Steve Atherton f0d35e87ef Renamed knob and URL parameter for request timeout to request timeout min to more accurately reflect what they do. 2020-07-19 17:41:59 -07:00
Young Liu 21c1998cca Fix MaxTLogQueueSize Bug 2020-07-16 15:56:04 -07:00