Commit Graph

209 Commits

Author SHA1 Message Date
Young Liu 60ccaa2ae9 Support finding max restorable version for key ranges when keyspace's max restorable version is unavailable 2020-09-17 17:58:58 -07:00
Young Liu d4e3e98957 Use new getRestoreSet in RestoreController 2020-09-17 17:56:36 -07:00
Meng Xu f9a5b727a6 Add comments questions and TODOs 2020-09-14 16:28:01 -07:00
Young Liu 3728ed03dd Resolve comments 2020-09-05 18:55:09 -07:00
Young Liu e9d1f1c9c8 change formatting 2020-08-31 09:43:11 -07:00
Young Liu b6c0299d09 Add help message in backup CLI for added options 2020-08-31 09:31:57 -07:00
Young Liu 30e27ba27b Add support for keys in CLI 2020-08-30 00:44:17 -07:00
Young Liu 33aa10b461 Minor optimizations 2020-08-29 20:10:45 -07:00
Young Liu fd7198d874 Extend backup container interface to support query restorable files set by key ranges 2020-08-29 19:58:07 -07:00
Balachandar Namasivayam 9c569c6ecf
Merge pull request #3569 from xumengpanda/mengxu/fr-fix-frtool-PR
FastRestoreTool: Fix --dest_cluster_file option
2020-07-28 09:56:00 -07:00
Meng Xu e232f805ee FastRestoreTool: Consume restoreClusterFileDest as fdbrestore does 2020-07-27 22:16:35 -07:00
Meng Xu 1a34f5c20e FastRestoreTool: Fix option list 2020-07-27 21:40:20 -07:00
Evan Tschannen e0db748fb3
Merge pull request #3403 from satherton/tls-background-handshake
TLS handshaking in background threads
2020-07-27 10:55:00 -07:00
Steve Atherton 38cade1146 Merged tag release-6.3.3 into tls-background-handshake. 2020-07-17 23:39:57 -07:00
Meng Xu 49095160e3
Merge pull request #3267 from Nishikoh/rename
Rename fastRestore_agent to fastRestoreTool
2020-07-16 15:33:07 -07:00
Oleg Samarin 25a61bc15d Detach a Clone of DR Site without Affecting Replication to the Original DR Site
Added description of fdbdr --dstonly to the usage text
2020-07-10 15:14:27 +03:00
Oleg Samarin 4c9df78076 dstonly 2020-07-10 15:13:42 +03:00
KoukiNishihara ab3de8700e Rename fastRestoreAgent to fastRestoreTool 2020-07-03 00:16:39 +09:00
Steve Atherton 648c870fec Merge branch 'release-6.3' of https://github.com/apple/foundationdb into tls-background-handshake 2020-06-26 00:21:57 -07:00
Steve Atherton 0da4c91ad5 Performance improvements in socket use and buffering which results in TLS network connections using around 3.5x less CPU.
PacketBuffers are 16k minimum sizes, using FastAlloc.  All calling paths to PacketBuffer creation now pass a size hint if known.  P2PNetworkTest improved in many ways, errors go to TraceEvents instead of stdout, error counts are reported in stats, range min/max parameters are combined into "min:max" strings, new options to "stutter" connections by delaying randomly before socket read or write, message size to send is now configured separately for incoming vs outgoing sessions.
2020-06-25 20:44:43 -07:00
Meng Xu c13ba7aeb4 FastRestore:Cosmetic code style improvement
Also mute some trace events
2020-06-24 21:25:37 -07:00
Meng Xu 2fcef90cbc FastRestore:Add addPrefix and removePrefix feature 2020-06-18 11:58:42 -07:00
Jingyu Zhou dc70591859 Fix fdbbackup help message 2020-06-11 17:26:21 -07:00
Evan Tschannen 3b9e9a9257
Merge pull request #3210 from ajbeamon/remove-include-version-h
Remove IncludeVersion.h
2020-05-20 14:12:23 -07:00
A.J. Beamon 4b5e94cf27
Merge pull request #3208 from apple/release-6.2
Merge release-6.2 into release-6.3
2020-05-20 13:47:48 -07:00
A.J. Beamon b589c3c940 Remove IncludeVersions.h, which is no longer needed for cmake-only builds 2020-05-20 13:23:02 -07:00
A.J. Beamon 54c97fe2bc Backup's ProgramStart command line string would be incomplete if arguments were specified using '=' 2020-05-20 09:38:33 -07:00
A.J. Beamon 59e33968c6 Remove erroneous restore options. 2020-05-20 08:57:56 -07:00
Evan Tschannen f17f00fdd5 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2020-05-10 22:33:38 -07:00
Meng Xu c237ca9708
Merge pull request #3081 from jzhou77/backup-cmd
Update fdbbackup documentation on new "-p" option
2020-05-04 14:17:34 -07:00
Jingyu Zhou 2c5da1c116 Remove "-p" option and use "--partitioned_log_experimental" option 2020-05-04 13:45:54 -07:00
Alex Miller 78d644f673
Merge branch 'master' into bugfixes/cmake-includes 2020-05-04 02:11:46 -07:00
Meng Xu 37a537c2a3 Fix MacOS compilation error 2020-04-30 18:30:15 -07:00
Steve Atherton 02aeab1edd Re-add improperly named options in order to not break compatibility with anything depending on them in a patch release. 2020-04-30 16:23:30 -07:00
Meng Xu 07a9a05683 FastRestore:Agent:Fix restore requests 2020-04-30 16:20:20 -07:00
Steve Atherton 697beb3ebb Fix fdbrestore parameters with one leading hyphen instead of two, which did not match documentation. 2020-04-30 12:13:35 -07:00
Meng Xu 8742cc0ab6 FastRestore:Fix windows build 2020-04-28 19:12:27 -07:00
Meng Xu f5e8345496 FastRestoreAgent:Use atomicParallelRestore to kick off restore
Replace the handcrafted version with atomicParallelRestore actor
which is simulation tested
2020-04-27 22:15:00 -07:00
tclinken 52860043c9 Merge remote-tracking branch 'origin' into atomically-update-dependent-knobs 2020-04-08 12:26:21 -07:00
Jingyu Zhou 9fb3fb9d82 Add pause/resume for new backups
To pause/resume the backup workers, the fdbbackup command will write to the
backupPausedKey. Then backup workers noticed the value of the key has been
changed and stops/resumes pulling from TLog.
2020-04-06 14:29:46 -07:00
Markus Pilman e4611e8ae4 fix versions.h stupidity 2020-04-06 10:28:55 -07:00
Markus Pilman 8b5780c36c don't include source and binary dir
This forces users to use include paths from the sources root.

So `#include "Arena.h"` won't work anymore, only
`#include "flow/Arena.h"` will.
2020-04-06 10:13:49 -07:00
tclinken 884e92bb49 Atomically update dependent knobs 2020-04-01 15:18:49 -07:00
Jingyu Zhou 906174e3e8 Add an option for fdbbackup to use new backup system
I.e., "-p", or "--partitioned_log" to enable it. By default, old backup system
is used.
2020-03-31 14:54:32 -07:00
Meng Xu a93f13cfd7 Remove redundant restoreRequestDone break in backup.actor 2020-03-25 15:19:46 -07:00
Meng Xu 495afe2e0b Improve how to wati for restore to finish
Remove default parameter for atomicRestore as suggested in review.
2020-03-25 13:54:21 -07:00
Evan Tschannen 303df197cf Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	bindings/c/test/mako/mako.c
#	documentation/sphinx/source/release-notes.rst
#	fdbbackup/backup.actor.cpp
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/NativeAPI.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/LogRouter.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/flow.vcxproj
#	flow/flow.vcxproj.filters
#	versions.target
2020-03-06 18:22:46 -08:00
Alex Miller 595dd77ed1 Merge remote-tracking branch 'upstream/release-6.2' into certificate-refresh 2020-03-04 20:25:42 -08:00
Alex Miller 9b5ef3416e Refactor TLSParams into TLSConfig + LoadedTLSConfig
The idea being that we keep around a TLSConfig that the configuration
that the user has provided, and then when we want to intialize an SSL
context, we ask the TLSConfig to load all certificates and return us a
LoadedTLSConfig that is a concrete set of certificate bytes in memory.

initTLS now just takes the in-memory bytes and applies them to the ssl
context.

This is a large refactor to lead up into certificate refeshing, where we
will periodically check for changes to the certificates, and then
re-load them and apply them to a new SSL context.
2020-03-04 20:14:47 -08:00
Evan Tschannen cdcb816866
Update fdbbackup/backup.actor.cpp 2020-03-04 16:08:45 -08:00