Commit Graph

209 Commits

Author SHA1 Message Date
Vishesh Yadav 2cd3f45fd6 Merge remote-tracking branch 'apple/release-6.3' into master-format-final
release-6.3 was recently merged, and there were two PRs which were
merged in between and got those changes in here. Hence, since all the
changes were in, discarded the incoming changes and accepted all
current.
2021-03-10 16:50:44 -08:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
FDB Formatster 8a8c488ede apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-05 18:13:38 -06:00
Andrew Noyes e32a0fb742 Resolve conflicts 2021-01-14 20:13:29 +00:00
Andrew Noyes 6d6b2843da Merge remote-tracking branch 'upstream/release-6.2' into anoyes/merge-release-6.2
I'm including the conflict markers for now and I'll resolve them in a
subsequent commit
2021-01-14 17:20:11 +00:00
Steve Atherton a860778b51 Add write buffering to BackupContainerLocalDirectory::BackupFile to greatly reduce the number of IAsyncFile::write() calls made when writing backup data. The buffer size is controlled by a knob. 2021-01-09 07:57:48 -08:00
sfc-gh-tclinkenbeard 4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
sfc-gh-tclinkenbeard 55e86c1336 Disambiguate between S3BlobStore and other blob stores 2020-10-29 20:42:23 -07:00
sfc-gh-tclinkenbeard e389a5dabe Fix issues with Azure credentials 2020-10-24 16:48:05 -07:00
sfc-gh-tclinkenbeard c6739f7075 Add skeleton for BackupContainerAzureBlobStore::listURLs 2020-10-24 16:48:05 -07:00
sfc-gh-tclinkenbeard f3c52ed41e Guard Azure backup compilation behind CMake variable 2020-10-24 16:48:04 -07:00
sfc-gh-tclinkenbeard e3f94c80fc Added BackupContainerAzureBlobStore::getURLFormat 2020-10-24 16:48:03 -07:00
sfc-gh-tclinkenbeard 6a27619544 Remove unnecessary copies in BackupContainer code 2020-10-24 16:48:03 -07:00
sfc-gh-tclinkenbeard ae91cf1d44 Make BackupContainerFileSystem.h a non-actor file 2020-10-24 16:48:02 -07:00
sfc-gh-tclinkenbeard e1d58d3c66 Moved BackupContainerS3BlobStore into its own files 2020-10-24 16:48:02 -07:00
sfc-gh-tclinkenbeard 961aeaecac Moved BackupContainerAzureBlobStore into its own files 2020-10-24 16:48:01 -07:00
sfc-gh-tclinkenbeard e5338d213b Moved BackupContainerLocalDirectory into its own files 2020-10-24 16:48:01 -07:00
sfc-gh-tclinkenbeard cd50d53b1e Created BackupContainerFileSystem.actor.h 2020-10-24 16:48:01 -07:00
sfc-gh-tclinkenbeard 29d6fc5dff Clarify scope of IBackupFile_impl namespace 2020-10-24 16:48:00 -07:00
sfc-gh-tclinkenbeard 0a99438d84 Update pNumDeleted in BackupContainerAzureBlobStore::deleteContainer 2020-10-24 16:48:00 -07:00
sfc-gh-tclinkenbeard ad911f4d61 Encapsulate some methods in BackupContainerAzureBlobStore 2020-10-24 16:47:59 -07:00
sfc-gh-tclinkenbeard 82b6daa16b First draft of Azure blob storage backup container 2020-10-24 16:47:51 -07:00
sfc-gh-tclinkenbeard a9607bdcec Explicitly seal classes that inherit but aren't inherited from 2020-10-07 21:58:24 -07:00
sfc-gh-tclinkenbeard a3de8c333b Use override where applicable in fdbclient 2020-10-07 16:57:38 -07:00
Meng Xu 1fb11858e6 Fix failure caused by merge 2020-09-27 11:03:16 -07:00
Meng Xu bbc7ce581e Resolve conflicts merging from 6.3 to master 2020-09-25 16:08:31 -07:00
Meng Xu 862336de8f Merge branch 'master' into mengxu/merge-to-master-PR 2020-09-24 17:06:00 -07:00
Young Liu d4e3e98957 Use new getRestoreSet in RestoreController 2020-09-17 17:56:36 -07:00
Young Liu 71309a79f9 Merge branch 'release-6.3' into range 2020-09-17 17:54:33 -07:00
Meng Xu 8224e17a08 Integrate extended getRestoreSet API into fast restore
The extended getRestoreSet provides a much smaller set of backup files for small keyrange restore;
This commit integrate it into fast restore so that fast restore does not have to filter out unneeded backup files.
2020-09-15 09:38:43 -07:00
Meng Xu f9a5b727a6 Add comments questions and TODOs 2020-09-14 16:28:01 -07:00
Meng Xu a3c861b59d
Merge pull request #3777 from apple/release-6.3
Merge Release 6.3 into master
2020-09-14 10:32:50 -07:00
Meng Xu 5880bd9fa7 BackupContainer:Fix getSnapshotFileKeyRange_impl on blob http error 2020-09-09 08:47:43 -07:00
Young Liu 3728ed03dd Resolve comments 2020-09-05 18:55:09 -07:00
Young Liu 1ad5e17458 add support for comparing original and current impls 2020-09-05 11:14:59 -07:00
Jon Fu b9baa996e3 Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only 2020-09-01 14:38:02 -04:00
Evan Tschannen 12edadd059 Merge branch 'release-6.3'
# Conflicts:
#	CMakeLists.txt
#	fdbclient/Knobs.cpp
#	fdbclient/MasterProxyInterface.h
#	fdbrpc/simulator.h
#	fdbserver/MasterProxyServer.actor.cpp
#	tests/fast/CycleAndLock.txt
#	tests/fast/TxnStateStoreCycleTest.txt
#	tests/fast/VersionStamp.txt
#	tests/slow/ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt
#	tests/slow/ParallelRestoreOldBackupCorrectnessCycle.txt
#	versions.target
2020-08-31 19:33:34 -07:00
Young Liu b6c0299d09 Add help message in backup CLI for added options 2020-08-31 09:31:57 -07:00
Young Liu 33aa10b461 Minor optimizations 2020-08-29 20:10:45 -07:00
Young Liu fd7198d874 Extend backup container interface to support query restorable files set by key ranges 2020-08-29 19:58:07 -07:00
Jon Fu 00c77ba2b4 Added beginVersion cmd line option and addressed code review comments 2020-08-28 14:29:22 -04:00
Meng Xu ca9b1f5b34 Merge branch 'release-6.3' into mengxu/fr-sched-PR
Resolve conflict at BackupContainer.actor.cpp
2020-08-27 16:54:00 -07:00
Meng Xu 369000a125 BackupContainer:Remove link to filename with random string 2020-08-26 15:55:36 -07:00
Meng Xu a2ab709a0c BackupContainer:Use processId as the process filename
instead of using a randomly generated string which change every time
when a file is open.

Having too many files will trigger TOO_MANY_FILES error
2020-08-26 15:54:34 -07:00
Meng Xu 6256bedf8d BackupContainer:Use processId as the process filename
instead of using a randomly generated string which change every time
when a file is open.

Having too many files will trigger TOO_MANY_FILES error
2020-08-25 12:25:09 -07:00
Jon Fu ae999aa118 Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only 2020-08-19 16:36:47 -04:00
Jon Fu 7dce3a9187 fixed issue with mutations not applying and allow backup to non-empty db 2020-08-11 15:39:21 -04:00
Jon Fu 21635f8a28 update backup restore for local testing 2020-08-04 15:48:43 -04:00
Evan Tschannen a49cb41de7 Merge branch 'release-6.3'
# Conflicts:
#	CMakeLists.txt
#	cmake/ConfigureCompiler.cmake
#	fdbserver/Knobs.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	flow/ThreadHelper.actor.h
#	flow/serialize.h
#	tests/CMakeLists.txt
2020-07-29 00:31:55 -07:00
Andrew Noyes d2cf700bd4 Fix compiler warnings 2020-07-28 18:30:26 +00:00