b495cc697b
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# versions.target
2019-09-13 09:25:08 -07:00
bd7678e71b
Remove --object-serializer help text
2019-09-05 11:57:59 -07:00
c2355f721e
Merge branch 'master' into mengxu/performant-restore-PR
2019-09-04 17:11:42 -07:00
d160810662
FastRestore:Resolve review comments
2019-09-04 16:48:43 -07:00
e465cbef07
Merge branch 'master' into remove-dl-iterate-phdr-hack
...
# Conflicts:
# fdbserver/fdbserver.actor.cpp
2019-08-22 10:13:23 -07:00
f02799455e
Add --loggroup to fdbserver and fdbbackup help text.
2019-08-19 12:59:14 -07:00
7ff46e6772
Merge branch 'master' into mengxu/performant-restore-PR
2019-08-07 20:31:56 -07:00
370ba8b841
Remove --object-serializer flag from executables
2019-08-06 09:25:40 -07:00
3b54363780
FastRestore:Apply Clang-format
2019-08-01 18:09:12 -07:00
7ccaeddf05
Merge branch 'master' into mengxu/performant-restore-PR
2019-08-01 13:23:17 -07:00
b5d2234a13
Merge branch 'release-6.1' into merge-release-6.1-into-master
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbserver/MoveKeys.actor.cpp
# flow/FastAlloc.h
# versions.target
2019-07-30 16:23:42 -07:00
9e1042d903
Bug fix, restore with --dryrun option would still require the --dest_cluster_file option even though it does not connect to a cluster.
2019-07-29 16:21:12 -07:00
b644f15b87
Bug fix: fdbrestore commands other than "start" were using default cluster file argument handling (but without the -C flag) instead of using the --dest_cluster_file argument.
2019-07-29 13:19:28 -07:00
b0c31f28af
FastRestore:Fix bug that blocks restore
...
1) Should recruit only configured number of roles;
2) Should never register a restore master interface as a restore worker (loader or applier) interface.
2019-07-25 17:55:37 -07:00
45083edf74
Merge branch 'master' into mengxu/performant-restore-PR
...
Fix conflicts as well.
2019-07-25 10:46:11 -07:00
1ac2d01b03
Merge remote-tracking branch 'upstream/master' into flatbuffers-fixes2
2019-07-18 09:50:08 -07:00
d5caf0c1b4
Merge branch 'flatbuffers-fixes2' of github.com:mpilman/foundationdb into flatbuffers-fixes2
2019-07-16 14:47:40 -07:00
d5051b08dd
Make trace event field lengths (and total event sizes) default knobified and configurable. Add a transaction option to control the field length of transaction debug logging. Make the program start command line field less likely to be truncated.
2019-07-12 16:12:35 -07:00
5be0c47409
Add some missing help text
2019-07-12 10:46:46 -07:00
269f077196
Add --object-serializer flag to fdbbackup etc
2019-07-12 10:46:45 -07:00
a5a6f8431c
Add a random UID to TransactionMetrics in case a client opens multiple connections and also a field to indicate whether the connection is internal. Convert some of the metrics to our Counter object instead of running totals.
2019-07-08 14:01:04 -07:00
6a899ddff3
Remove dl_iterate_phdr results caching that is used by slow task profiler, instead favoring disabling and reenabling profiling around the call. Add a mechanism to handle deferred profile requests.
2019-07-03 12:48:36 -07:00
37c1df2491
Merge pull request #1705 from bnamasivayam/suspend-process
...
Extend RebootRequest API to include time to suspend the process befor…
2019-06-20 17:36:25 -07:00
68ce9a5e75
ProtocolVersion type - second try
2019-06-18 17:55:27 -07:00
20e3edeb0a
Merge branch 'release-6.1'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbserver/storageserver.actor.cpp
# versions.target
2019-06-14 12:42:59 -07:00
5eb833759e
Extend RebootRequest API to include time to suspend the process before reboot. This is intended to be used for testing purposes to simulate failures.
2019-06-14 11:35:38 -07:00
235d24ee26
Bug fix: fdbrestore abort, wait, and status were not using the --dest_cluster_file argument.
2019-06-13 17:42:11 -07:00
3fcb6ec0a1
FastRestore:Refactor RestoreLoader and fix bugs
...
Refactor RestoreLoader code and
Fix a bug in notifying restore finish.
2019-06-04 21:53:31 -07:00
477fd152c0
FastRestore:Refactor code
...
1) Use the runRYWTransaction for simple DB access
2) Replace some printf with TraceEvent
3) Remove printf not used in debugging
4) Avoid wait inside the condition in loop-choose-when for
the core routine of restore worker, loader and applier.
5) Rename Restore.actor.cpp to RestoreWorker.actor.cpp since
the file only has functionalities related to restore worker.
Passed correctness test
2019-06-04 11:22:47 -07:00
a3f61e6df7
FastRestore:Rafctor:Reduce code size
...
1) Use runRYWTransaction to replace the loop-try style;
2) Remove unnecessary printf
3) Do not mistakenly send reply twice.
2019-05-29 17:03:50 -07:00
5f55f3f613
Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used.
2019-05-10 14:01:52 -07:00
a43071339c
fdbbackup: fix some print/scan format warnings
...
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-05-06 13:35:29 -07:00
705b1d1538
FastRestore: Reduce the printf further
...
This help reduce the false positive in correctness.
2019-04-19 09:59:12 -07:00
529ce66b6c
Merge branch 'apple/master' into mengxu/performant-restore-PR
2019-04-18 18:02:45 -07:00
ef04471a66
Fix more unused-variable warnings
2019-04-17 16:04:10 -07:00
4c3ccebe8a
FastRestore: Cleanup code
...
Remove unused code and comments.
2019-04-12 13:49:55 -07:00
1c16f87a4e
Remove trace-calls to printable (in non-workloads)
2019-04-05 13:12:19 -07:00
d68c9ec09e
FastRestore: Fix after merge with master
2019-03-31 22:07:37 -07:00
70d7c289f4
Merge branch 'master' into mengxu/restore/parallel-v7
2019-03-30 22:13:10 -07:00
589fb76826
FastRestore:Attempt to fix old restore
2019-03-30 15:19:30 -07:00
7c02ee6fdd
Fix compiler warning about unreferenced exception variable
2019-03-26 13:43:47 -07:00
56955deed9
Set TRACE_FORMAT network option in fdbbackup
2019-03-22 10:39:13 -07:00
8aab719c22
Merge branch 'master' into feature-backup-json
2019-03-12 18:23:16 -07:00
bc0b2aa040
Merge branch 'release-6.0' of https://github.com/apple/foundationdb
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbclient/BlobStore.actor.cpp
2019-03-12 04:49:12 -07:00
f0eae0295f
Merge branch 'master' of https://github.com/apple/foundationdb into feature-backup-json
2019-03-12 03:35:03 -07:00
041b261464
Remove short option name for active snapshot interval
2019-03-11 01:48:51 -07:00
023bbb566f
Renamed backup state enums for clarity, added backup state names. Changed Epochs to EpochSeconds in backup JSON along with some other renaming/moving of fields, and added information about snapshot dispatch. Changed timestamp format for input/output in all backup/restore contexts to be a fully qualified time with timezone offset. Added information about the last snapshot dispatch to backup config and status (not yet populated).
2019-03-10 16:00:01 -07:00
ebffe8c633
print correct pahes in alloc instrumentation
2019-03-08 15:03:17 -08:00
1399aee532
Added --json option to fdbbackup status.
2019-03-06 21:32:46 -08:00
ca8bbad657
Added --json option to fdbbackup describe. Also added expired percentage indicator to snapshot details.
2019-03-06 14:14:06 -08:00
7778112f6a
Bug fix, restore was using the destination cluster to look up timestamps when printing the backup description instead of (optionally) the original cluster which generated the backup. Made missing cluster file errors more clear.
2019-03-06 02:45:55 -08:00
888c4764d3
Bug fix: fdbrestore was not using a default range set.
2019-03-06 01:57:03 -08:00
28e67aa041
Fixed option name to match documentation.
2019-03-05 23:26:45 -08:00
eed8dc816d
Bug fix: fdbbackup modify on a tag that has never been used before would result in an internal error.
2019-03-05 22:17:37 -08:00
7f37e25c75
Fdbbackup binary will ignore .debug suffix when determining how it was invoked.
2019-03-05 21:17:56 -08:00
a57ddd29f2
Bug fixes in fdbbackup modify command. Active snapshot interval was wrong. New backup destination containers were not being created. Renamed some variables for clarity.
2019-03-05 18:05:52 -08:00
337a5aa1fe
Updated to new Database API.
2019-03-05 12:44:28 -08:00
2f7d559442
Restore now requires an explicit destination cluster file. Restore target database version can now be specified by a timestamp, which must be resolved using the cluster file for the cluster from which the backup originated.
2019-03-05 12:28:05 -08:00
a5b53d3c0f
More documentation and release note changes for fdbbackup modify. Added release note for #1205 bug fix which was missed.
2019-03-05 10:44:14 -08:00
21f55e1878
Merge pull request #1190 from bnamasivayam/restore-multiple-ranges
...
Add support for restoring multiple ranges.
2019-03-05 10:15:55 -08:00
f6559bbb11
Added help and documentation for fdbbackup modify. Add BackupUID and more parseable BackupURL to fdbbackup status output.
2019-03-05 05:13:48 -08:00
d3377722d5
Added blob store Backup URL parameter 'header' which enables addition of custom HTTP header fields to blob store HTTP requests. Added 'fdbbackup modify' command line tool for changing the backup URL and parameters, default snapshot interval, and/or current snapshot interval of a running backup.
2019-03-05 04:00:11 -08:00
627edd4724
Add force option to 'fdbdr switch'.
2019-03-04 16:36:58 -08:00
a258df32f6
Skip switchover checks for force option.
2019-03-04 15:58:36 -08:00
7cf71b0931
Add some basic checks before doing an atomic switchover.
2019-03-01 14:49:04 -08:00
7eba50b086
Add support for restoring multiple ranges.
2019-02-25 18:00:28 -08:00
78dd80ea8a
Proper fwd decl in BackupAgent
...
Also BackupAgent.h -> BackupAgent.actor.h
2019-02-19 15:16:59 -08:00
067a445e06
Replace unused _ variables with wait(success(...))
2019-02-12 17:30:30 -08:00
6da5971e79
Guard all versions.h to not break old WIN32 build
2019-02-08 16:06:00 -08:00
7e26b4ef0d
Address comments from PR
2019-02-07 15:37:04 -08:00
5737349676
Fix weird bug with boost interprocess
...
Strangely, boost interprocess didn't compile with VS 2017.
However, it does compile if it is included as the first thing.
I don't quite know what is happening here, but for now this fix
makes it that I am not blocked
2019-02-07 15:37:04 -08:00
f0a58c6e31
Corrected spacing.
2019-02-06 11:35:50 -08:00
b1e3c9c22d
Corrected spacing issue.
2019-02-06 11:27:26 -08:00
d1145e5c04
Support TLS command line options in backup
2019-01-30 17:36:51 -08:00
550f2e2682
Merge with master to use the latest backup container
...
In fdb 6.0.15, backup container is changed on how to organize the backup data.
The backup made by fdb >6.0.15 has to be restored with fdb > 6.0.15.
Merge with master so that the fast restore uses fdb > 6.0.15
2019-01-30 12:05:15 -08:00
2e11b38f3f
Add print in fast restore agent about backup info
2019-01-30 11:18:11 -08:00
4e9dcb3c74
fastrestore_agent must use lower_case name
2019-01-25 14:01:09 -08:00
a41bcf79d8
Move fastRestore function from fdbserver into fdbbackup
...
fdbbackup cannot depend on fdbserver.
So we cannot use functions defined in fdbserver in fdbbackup or fdbclient.
We move the fastRestore function that initialize the restore request from fdbserver to fdbclient.
It is compilable now.
2019-01-24 19:25:14 -08:00
16f363b234
Add fdbserver dependency to fdbbackup
2019-01-24 18:00:18 -08:00
e721a6a99e
FastRestore:Add restore agent
...
Restore agent will by used by system operator to send out the restore requests to restore workers
Restore agent uses the same code as backup agent, except that its binary name is fastRestore_agent
2019-01-24 16:26:59 -08:00
684a22a52b
Merge branch 'release-6.0'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbclient/BackupContainer.actor.cpp
# fdbclient/HTTP.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/BackupCorrectness.actor.cpp
# versions.target
2019-01-09 16:14:46 -08:00
f116678f0d
Bug fix, restore start dry run would not open a trace file.
2019-01-09 03:54:13 -08:00
e7f800b019
Restore start with dry run option no longer requires or uses a database. Removed duplicate and unnecessary code. Dry run execution is more clear, it does not pretend to have actually restored data. Fixed typo in help text.
2019-01-08 19:00:58 -08:00
276ca82af4
Added --loggroup option to all fdbbackup modes.
2019-01-04 00:15:51 -08:00
71f9de2db6
Fix whitespace
2019-01-03 09:12:03 -08:00
7eb6765698
Mention that xml is the default
2019-01-03 08:48:31 -08:00
19c0ccd4c0
Added fdbbackup dump option to list files and sizes in a backup for a given version range.
2019-01-03 02:35:31 -08:00
bce5b03340
Fix whitespace
2019-01-02 15:24:11 -08:00
f64d5321e9
Backup describe, expire, and delete will now clearly indicate when there is no backup at the given URL.
2018-12-20 18:05:23 -08:00
1bca665b29
Document --trace_format flag
2018-12-20 16:22:41 -08:00
03b48fb452
s/--trace-format/--trace_format/
2018-12-20 15:58:26 -08:00
b8df5acc15
Add --trace_format flag to fdbserver
2018-12-20 15:02:01 -08:00
567a7bd58a
Print a final deleted object count at the end of backup deletion.
2018-12-20 01:24:31 -08:00
354abebf64
Added progress reporting to backup expiration. Simplified backup delete progress.
2018-12-20 00:23:26 -08:00
afa243bc97
Added fdbbackup expire options to calculate approximate version boundaries based on a number of days prior to the latest log file found in the backup container. This enables expiration operations based on time (with reasonable precision) without accessing the source cluster.
2018-12-18 18:55:44 -08:00
779a4b1e3b
Static de-initialization order isn't well thought out so use flushAndExit() instead of returning from main after runNetwork() has been called.
2018-12-17 18:39:14 -08:00
4b5d0b4e2c
Merge branch 'release-6.0'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbclient/AsyncFileBlobStore.actor.cpp
# fdbclient/AsyncFileBlobStore.actor.h
# fdbclient/BlobStore.actor.cpp
# fdbclient/BlobStore.h
# fdbclient/HTTP.actor.cpp
# fdbclient/ManagementAPI.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbrpc/LoadBalance.actor.h
# fdbrpc/batcher.actor.h
# fdbrpc/fdbrpc.vcxproj
# fdbrpc/sim2.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/DataDistributionTracker.actor.cpp
# fdbserver/SimulatedCluster.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/masterserver.actor.cpp
2018-11-10 13:04:24 -08:00
a654183f63
Merge pull request #791 from ajbeamon/remove-cluster-from-iclientapi
...
Remove cluster from IClientApi (phase 2 of removing DB names)
2018-11-10 10:16:18 -08:00
776b289bfe
Move AsyncFileBlobStore and related files to fdbclient.
2018-10-26 13:49:42 -07:00
0217aed74c
Merge branch 'release-6.0'
...
# Conflicts:
# bindings/go/README.md
# documentation/sphinx/source/release-notes.rst
# fdbserver/MasterProxyServer.actor.cpp
# versions.target
2018-10-15 18:38:51 -07:00
Evan Tschannen