Commit Graph

346 Commits

Author SHA1 Message Date
RenxuanW c8b27e71c5 Revert TraceEvent
We've found the problem (issue #4640), so we no longer need the TraceEvent.
2021-04-12 15:03:47 -07:00
RenxuanW dc00d99626 Log FileBackupLogRangeStart before calling getLogRanges() in .
It will tell us if or why this function is legitimately trying to use too much ram. getLogRange() should normally return about 20 items in the result. If the inputs are trash, it could return far more.

If it isn’t the case, then there’s something else wrong that has corrupted something such that when we try to allocate memory.
2021-04-09 15:02:45 -07:00
RenxuanW fadc9cccee Use knob RESTORE_IGNORE_LOG_FILES in restore.
Rename IGNORE_LOG_FILES to RESTORE_IGNORE_LOG_FILES. Also, this knob should be used in regular restore, not parallel restore.
2021-04-07 10:05:56 -07:00
RenxuanW 1b6ad42db8 Use a knob to completely ignore log files 2021-04-06 19:07:01 -07:00
Jingyu Zhou 2b4744ad8a Reword comments. 2021-03-29 13:43:43 -07:00
Jingyu Zhou 55ef40864f Add comments for touched functions 2021-03-29 10:09:07 -07:00
Jingyu Zhou 1a0ef39e5d Fix keyRangesFilter with logsOnly flag
Remove the Sev 40 trace event.
2021-03-27 21:48:27 -07:00
Jingyu Zhou a9d8806828 Use the restored range in the actual restore 2021-03-25 10:07:25 -07:00
Steve Atherton 720c0b3a4f
Added const.
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2021-03-24 14:36:55 -07:00
Steve Atherton a6136ffc62 Applied clang-format to recent changes. 2021-03-24 06:51:08 -07:00
Steve Atherton 17a19021fb Added mutation batch filtering during restore, to the extent possible where each restore mutation log task only sees one block of data (usually 1MB) which can contain incomplete and therefore unfilterable mutation sets. 2021-03-24 06:38:50 -07:00
A.J. Beamon 3751ecd11f
Merge pull request #4503 from sfc-gh-ajbeamon/merge-release-6.3-into-master
Final merge of release-6.3 into master
2021-03-16 09:29:05 -07:00
A.J. Beamon 25c4880ebe Merge branch 'release-6.3' into merge-release-6.3-into-master (temporarily discard all changes to BackupContainer.actor.cpp)
# Conflicts:
#	fdbclient/BackupContainer.actor.cpp
#	fdbserver/Knobs.h
2021-03-15 16:41:04 -07:00
Trevor Clinkenbeard 766b9d24d6
Merge branch 'master' into remove-invalid-backup-status-fields 2021-03-14 18:57:03 -07:00
Markus Pilman cfbaaa6001 Merge remote-tracking branch 'origin/release-6.2' into release-6.3 2021-03-10 14:30:47 -07:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
sfc-gh-tclinkenbeard f81d4eb6d3 Remove invalid backup status fields 2021-03-08 15:48:46 -08:00
Steve Atherton 318b862aa3 Apply clang-format to backup changes. 2021-03-08 01:49:29 -08:00
Steve Atherton 8a4aca3f47 Bug fix: In simulation only, RangeFileWriter was not waiting on a write future in one location. 2021-03-07 22:19:04 -08:00
FDB Formatster 8a8c488ede apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-05 18:13:38 -06:00
A.J. Beamon aaf0a9aa7b Merge branch 'release-6.3' into merge-release-6.3-into-master
# Conflicts:
#	build/docker-compose.yaml
#	cmake/ConfigureCompiler.cmake
#	fdbclient/FileBackupAgent.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/IAsyncFile.h
#	fdbrpc/IRateControl.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbservice/ServiceBase.cpp
2021-02-08 12:58:34 -08:00
Christophe Chevalier 3fc8870ce3 winbuild: fix duplicate definition of Codec<ERestoreState>::pack and unpack in Debug build
- defined twice in FileBackupAgent.actor.cpp and BackupAgent.actor.h
- only fails when building in Debug ?
2021-01-30 19:03:25 +01:00
Andrew Noyes cec050a1f8 Apply clang-tidy's modernize-use-override fix 2021-01-26 02:04:12 +00:00
sfc-gh-tclinkenbeard 4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
Jon Fu 51db9a7e0a add static method to access backup pause key instead of constructing it manually 2020-11-06 14:03:29 -05:00
sfc-gh-tclinkenbeard c1d88b39ab Merge remote-tracking branch 'origin/master' into add-upgrade-and-backup-restore-test 2020-10-17 11:35:12 -07:00
Andrew Noyes a1e868a569
Merge pull request #3862 from sfc-gh-tclinkenbeard/use-override-more
Add uses of override keyword, remove unnecessary uses of virtual
2020-10-14 15:06:45 -07:00
sfc-gh-tclinkenbeard ccf2ca73f3 Fix Windows CMake build 2020-10-08 23:58:58 -07:00
sfc-gh-tclinkenbeard a9607bdcec Explicitly seal classes that inherit but aren't inherited from 2020-10-07 21:58:24 -07:00
sfc-gh-tclinkenbeard a3de8c333b Use override where applicable in fdbclient 2020-10-07 16:57:38 -07:00
Jon Fu 7218586f2f adjust test spec and backup describe 2020-10-06 17:03:09 -04:00
sfc-gh-tclinkenbeard 43ac7f21a4 Mark toString methods const 2020-10-04 23:27:10 -07:00
sfc-gh-tclinkenbeard 8ae0679112 Remove unnecessary separation of declaration and definition of version field 2020-10-04 23:27:05 -07:00
sfc-gh-tclinkenbeard 6326de052b Remove unused LogFileWriter::FFs 2020-10-02 17:03:08 -07:00
Jon Fu 8da8c6ff0d Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version 2020-09-30 14:25:29 -04:00
Jon Fu 44cd3b0999 add stopBackup to incrementalBackup workload 2020-09-30 14:24:52 -04:00
Jon Fu cfaa207f88 fix spacing and newlines at ends of files 2020-09-28 16:32:20 -04:00
Jon Fu 3aa4070adb remove trace events used for debugging 2020-09-28 16:27:30 -04:00
Meng Xu 1fb11858e6 Fix failure caused by merge 2020-09-27 11:03:16 -07:00
Meng Xu bbc7ce581e Resolve conflicts merging from 6.3 to master 2020-09-25 16:08:31 -07:00
Jon Fu 6ed330572e Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version 2020-09-24 12:09:17 -04:00
sfc-gh-tclinkenbeard d13a9d3c91 Make EBackupState an enum class 2020-09-23 15:14:09 -07:00
Jon Fu 69580593dd Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version 2020-09-23 15:35:05 -04:00
Jon Fu 9b7d75b6dd WIP 2020-09-21 13:41:35 -04:00
Meng Xu a3c861b59d
Merge pull request #3777 from apple/release-6.3
Merge Release 6.3 into master
2020-09-14 10:32:50 -07:00
Meng Xu f10e9ea679 FastRestoreApplier:Add write conflict range 2020-09-09 12:12:14 -07:00
Meng Xu e69ee87f33 Refactor blob failure to simulateBlobFailure function 2020-09-09 10:54:21 -07:00
Meng Xu 20733ae1eb Simulate throwing blob http error immediately after read blob file 2020-09-09 06:47:19 -07:00
Meng Xu cffb54ba0b Buggify decodeLogFileBlock with blob http error 2020-09-09 06:44:02 -07:00
Meng Xu f3986d201d Buggify backup decodeRangeFileBlock with blob connection error 2020-09-09 06:40:41 -07:00