Commit Graph

453 Commits

Author SHA1 Message Date
Xiaoxi Wang 2abdbff11f add Knobs 2021-07-12 05:33:23 +00:00
sfc-gh-tclinkenbeard 8cc40e3a2b Expand use of BOOLEAN_PARAM 2021-07-02 21:41:50 -07:00
Lukas Joswiak f9d7c874ac Fix invalid read 2021-07-01 21:40:02 -07:00
Neethu Haneesha Bingi decfb610ff Minor review comments. 2021-06-25 15:04:49 -07:00
Neethu Haneesha Bingi 66f2518405 exclude to work with any locality data match. 2021-06-23 18:03:27 -07:00
Neethu Haneesha Bingi cbe714acd0 Status json schema update, includelocalities back for consistency check, review comments. 2021-06-23 18:03:27 -07:00
Neethu Haneesha Bingi 4ad5926a25 Snake naming of keys and added comments to all new functions. 2021-06-23 18:03:27 -07:00
Neethu Haneesha Bingi 73752f441b exclude locality:clang-format, ranged loops, documentation, tracking addStoragesever for exclusion. 2021-06-23 18:03:27 -07:00
Neethu Haneesha Bingi 62355571d0 exclude servers based on locality match 2021-06-23 18:03:27 -07:00
sfc-gh-tclinkenbeard 371a38e6e5 Merge remote-tracking branch 'origin/master' into remove-extra-copies 2021-06-07 10:26:06 -07:00
Andrew Noyes fbe96939ba Avoid likely use after destruction
This manifested in UBSAN as use of an invalid vptr, which usually means
use after destruction. Give shared ownership to the getDesired actor to
avoid use after destruction. It is not clear to me when this doesn't
seem to manifest as a heap-use-after-free in e.g. ASAN or valgrind.
2021-06-04 09:54:06 -07:00
Josh Slocum d67184163b
Merge pull request #4556 from sfc-gh-jslocum/tss
Testing Storage Server
2021-06-01 09:11:10 -07:00
Evan Tschannen 4a753a0bce
Merge pull request #4623 from dlambrig/issue4252
CDs reject requests using different connection descriptor
2021-05-28 10:45:51 -07:00
Dan Lambright 10289ef8f1 Respond to AJs comments 2021-05-27 09:14:32 -04:00
Dan Lambright 64c10d3625 fix joshua failures, formatting 2021-05-27 08:08:07 -04:00
Dan Lambright fcfb78162c misc cleanup for publishing 2021-05-27 08:08:07 -04:00
Dan Lambright 742c22cef2 Don't allow changing desriptor if knob is set 2021-05-27 08:08:07 -04:00
Josh Slocum 4257ac2b4d More TSS Changes/Fixes 2021-05-25 20:37:48 +00:00
Josh Slocum ce82c9653e Testing Storage Server implementation 2021-05-25 20:28:50 +00:00
Xiaoxi Wang 8b1f9f7337 Add command line support 2021-05-17 20:57:54 +00:00
sfc-gh-tclinkenbeard f28ac955c3 Remove unnecessary temporary objects while growing objects of type std::vector<std::pair<A, B>> 2021-05-10 16:32:50 -07:00
sfc-gh-tclinkenbeard 5c2d7b6080 Create RangeResult type alias 2021-05-03 13:14:16 -07:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Andrew Noyes 79cec09255 Apply clang-tidy's performance-inefficient-vector-operation fix
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.

    $ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
2021-03-04 03:58:25 +00:00
negoyal e9b7e22e02 Merge branch 'rel-6.3' 2021-02-22 08:47:22 -08:00
negoyal e49cc68bbc Merge branch 'rel-6.2' into rel-6.3 2021-02-21 00:15:31 -08:00
Andrew Noyes 9fb396e372
Merge pull request #4255 from sfc-gh-clin/add-coordinators-into-special-keys
Add coordinators into special keys
2021-02-18 18:26:07 -08:00
A.J. Beamon af77fae131 Fix bug in code I recently added where the process name memory was not long-lived and was later used. 2021-02-18 18:33:23 +00:00
Andrew Noyes bcd9a00a50 Backport #4285 and 1513df22f3 to release-6.2
Fixes #4342
2021-02-17 00:35:25 +00:00
Chaoguang Lin 731ee8a121 Update ChangeConfig to test coordinators special keys 2021-02-16 13:01:37 -08:00
Steve Atherton 76a4293d1d Merge branch 'release-6.3'
# Conflicts:
#	fdbserver/fdbserver.actor.cpp
2021-02-15 02:13:06 -08:00
A.J. Beamon 5b74556733 Fix bug in code I recently added where the process name memory was not long-lived and was later used. 2021-02-10 10:46:02 -08:00
Chaoguang Lin 3def9731a4 Merge branch 'master' of github.com:apple/foundationdb into add-coordinators-into-special-keys 2021-02-08 18:52:45 -08:00
A.J. Beamon be57b70dbb Fix function call where the signature changed 2021-02-08 14:00:47 -08:00
A.J. Beamon aaf0a9aa7b Merge branch 'release-6.3' into merge-release-6.3-into-master
# Conflicts:
#	build/docker-compose.yaml
#	cmake/ConfigureCompiler.cmake
#	fdbclient/FileBackupAgent.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/IAsyncFile.h
#	fdbrpc/IRateControl.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbservice/ServiceBase.cpp
2021-02-08 12:58:34 -08:00
A.J. Beamon 84be77958e Add a simulator guard 2021-02-05 22:47:49 -08:00
A.J. Beamon 53d8bd5799 If a quorum change has to retry, it's possible that one of our previously chosen coordinators could be rebooting. 2021-02-05 22:36:54 -08:00
A.J. Beamon 5c4863b33e Fix unit test that set coordinators to fake processes that the simulator didn't know about 2021-02-05 22:07:40 -08:00
A.J. Beamon 618585f9dd Fix a bug in simulated coordinator selection that could allow an unreliable process to be chosen as coordinator despite the intent that this not happen. 2021-02-05 15:59:14 -08:00
Chaoguang Lin 766de6574a Add trace 2021-01-29 15:32:53 -08:00
Chaoguang Lin 5a62e0c084 Fix repeated code usage 2021-01-29 12:20:30 -08:00
Chaoguang Lin e1182f2f86 Add coordinators to special keys 2021-01-29 11:45:52 -08:00
David Youngworth d64cf8b9e3 Merge branch 6.3 into master 2020-11-17 11:22:45 -08:00
Jon Fu cc13ef08bd Sort the failed sets before modifying them in attempts to make changes consistent 2020-11-12 16:26:34 -05:00
sfc-gh-tclinkenbeard 0718329705 Make getDesiredClusterKeyName const
This fixes a virtual function overload in NameQuorumChange
2020-10-15 10:39:38 -07:00
sfc-gh-tclinkenbeard ccf2ca73f3 Fix Windows CMake build 2020-10-08 23:58:58 -07:00
sfc-gh-tclinkenbeard a9607bdcec Explicitly seal classes that inherit but aren't inherited from 2020-10-07 21:58:24 -07:00
sfc-gh-tclinkenbeard a3de8c333b Use override where applicable in fdbclient 2020-10-07 16:57:38 -07:00
Balachandar Namasivayam c7a55fe15f
Merge pull request #3829 from yliucode/process-class
Support specifying proxy as process class and converting it to commit proxy
2020-09-30 16:46:24 -07:00
Young Liu 0c579674e6 Add more help messages for configuring 'proxies' 2020-09-30 13:29:01 -07:00
Young Liu c34385ef7b Support proxy process class 2020-09-30 01:23:10 -07:00
sfc-gh-tclinkenbeard 6c726ba8dd Improve ConfigurationResult and CoordinatorsResult type safety 2020-09-27 15:29:15 -07:00
Young Liu 3278e21b0b Change description of GRV proxy and Commit proxy in docs and add backward compatibility support for configuring proxies in fdbcli 2020-09-25 10:50:48 -07:00
Meng Xu cf69f455a9
Merge pull request #3785 from apple/release-6.3
Merge Release 6.3 to master
2020-09-17 14:43:56 -07:00
Young Liu cc5bc16bd8 Rename more places from proxy to commit proxy 2020-09-15 22:29:49 -07:00
Young Liu 35bef73a1c Rename proxy to commit proxy 2020-09-10 17:44:15 -07:00
Young Liu 1155d015c9 fetch current log generation as well 2020-09-09 11:54:58 -07:00
Young Liu 79ce16650d merge master branch 2020-08-11 19:22:10 -07:00
Young Liu d6a23a4d6b Resolve comments to make GRV proxy a separate process class 2020-08-06 00:01:57 -07:00
Chaoguang Lin b682ad75f9 Fix a memory bug, change \xff\xff/management/options/exclude/force to .../excluded/force 2020-08-05 13:18:39 -07:00
Chaoguang Lin df4beb629e fix the bug where we call old \xff\xff/conf special keys 2020-07-31 13:28:24 -07:00
Chaoguang Lin db8bb69196 Solve conflicts from master 2020-07-30 19:02:19 -07:00
Chaoguang Lin f2662a9a74 Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space 2020-07-30 19:01:11 -07:00
Young Liu fad18ad85a Remove more debug traces 2020-07-29 08:06:15 -07:00
Young Liu 30ea639666 Remove debug traces 2020-07-29 07:55:05 -07:00
Young Liu f7b76a92af pass joshua 2020-07-29 07:26:55 -07:00
Meng Xu a2089b354a RemoveServersSafely:Safety check toKill1 to avoid cluster getting stuck
toKill1 and toKill2 are a random subset of all processes. If simply kill all processes in toKill1 or toKill2,
we may kill too many processes to make the cluster unavailable and stuck.

Similar as what toKill2 were modified if it can cause cluster unavailable,
we should do the same thing for toKill1
2020-07-28 21:07:31 -07:00
Chaoguang Lin 27a7a28fc5 Change to ryw in checkForExcludingServers 2020-07-28 13:40:14 -07:00
Chaoguang Lin 8a5fde1b85 change SPECIAL_KEY_SPACE_CHANGE_CONFIGURATION to SPECIAL_KEY_SPACE_ENABLE_WRITES 2020-07-28 12:03:00 -07:00
Young Liu 1826ac75d5 Add some trace events to debug 2020-07-25 18:16:08 -07:00
Young Liu ff4bae5cd3 Fix status test 2020-07-23 12:04:02 -07:00
sfc-gh-tclinkenbeard 0d4fd085d1 Add some extra braces to for and if statements 2020-07-21 09:19:19 -07:00
sfc-gh-tclinkenbeard abaae40453 Modified ComposedIdentifier to prevent file identifier conflicts at
compile-time
2020-07-20 18:52:44 -07:00
Chaoguang Lin 1e5096990e Add the function to return the special key for given management api command and option 2020-07-17 12:32:42 -07:00
Chaoguang Lin 96e5a4c827 Replace all \xff\xff/conf with managementApiRange.begin 2020-07-17 11:56:27 -07:00
Chaoguang Lin 5fa6785fb8 Add special keys for command options 2020-07-16 18:15:35 -07:00
Chaoguang Lin 3593ceffde Add back previous implementation code for versioning 2020-07-14 12:38:06 -07:00
Chaoguang Lin 1635e8e0ba using special keys in management api for include when api_version >= 700 2020-07-14 10:31:19 -07:00
Chaoguang Lin 4d7e573469 using special keys in management api for exclude when api_version >= 700 2020-07-14 09:52:38 -07:00
Chaoguang Lin b0b601478c Add TODO comments 2020-07-13 13:34:15 -07:00
Chaoguang Lin 2a434791ad Exposing transaction based function to get excluded and failed server addresses 2020-07-13 10:28:34 -07:00
Chaoguang Lin 407985e142 Revert separating excluding function into excluded and failed 2020-07-13 10:25:36 -07:00
Chaoguang Lin b8251878c4 Separating two functions for excluded and failed list. Exposing transaction based interface from Management Api 2020-07-12 10:41:46 -07:00
Chaoguang Lin 5f6093b59e Add an option to enable write in special key space, which is by default not allowed 2020-07-08 15:49:34 -07:00
Chaoguang Lin 784d0f6d00 A basic feasible version for exclude and include commands 2020-07-06 11:02:48 -07:00
Meng Xu 96206a8032
Merge pull request #3368 from apple/release-6.3
Merge Release 6.3 to master
2020-06-15 20:15:22 -07:00
Daniel Smith acbfe2e4c9
Revert "Revert "Initial RocksDB"" 2020-06-15 12:45:36 -04:00
Jingyu Zhou 9cd1614c82
Revert "Initial RocksDB" 2020-06-11 15:29:46 -07:00
negoyal cf13e00a8f Merge remote-tracking branch 'origin/release-6.3' into fdb_cache_wo_allocator 2020-06-01 17:38:31 -07:00
Daniel Smith 8731700d80 Merge remote-tracking branch 'upstream/release-6.3' into rocksdb-6.3 2020-05-27 20:02:25 +00:00
Evan Tschannen ced65cd30b finished explicitly versioning everything stored in the database 2020-05-22 17:14:21 -07:00
Daniel Smith 5d361fe532 Copy/paste rebase onto 6.3 2020-05-22 15:02:51 +00:00
tclinken 3a8efb2d0b Added advanceversion command to fdbcli 2020-05-20 15:47:31 -07:00
negoyal dd033736ed Merge branch 'master' into fdb_cache_subfeature2 2020-05-04 17:29:43 -07:00
tclinken 93400d25c8 Added advanceversion command to fdbcli 2020-04-15 20:01:01 -07:00
Neelam Goyal 927282771a
Update fdbclient/ManagementAPI.actor.cpp
Co-Authored-By: Trevor Clinkenbeard <trevorclinkenbeard@gmail.com>
2020-04-13 12:17:54 -07:00
negoyal acaf91ac47 Merge branch 'master' into fdb_cache_subfeature2 2020-03-26 13:33:08 -07:00
negoyal abd03ba998 Clear the range correctly when adding/deleting a cache range. 2020-03-17 17:13:44 -07:00
Meng Xu 7f559bc712 Cleanup code and apply clang-format
Self code review
2020-03-16 15:08:32 -07:00
Meng Xu 1513df22f3 AutoQuorumChange:Exclude unreliable node from coordinator in simulation 2020-03-16 14:39:25 -07:00
Meng Xu 15c48b9e19 Add event for getDesired coordinators 2020-03-16 09:40:35 -07:00
Meng Xu bd345f85db ConsistencyCheck:Fix failue due to address inconsistency between process and worker
With TLS, a worker (or process) can have a TLS address and non-TLS address.
When a process is created in simulation, the primary address is TLS by default.
The non-TLS one is the TLS address port plus one.

In a connection between two workers, if their primary addresses do not enable
or disable TLS together, one worker will swap its primary address and secondary address
so that the TLS config of the two endpoints can match.

The swap can make the primary address no longer the TLS one that was created
when the process is created. And the swap only happens for worker instead of
process struct in simulation.

This swap can cause worker->address != process->address.
In checkForExtraDataStores actor, we use worker->address to check if a process
is killable and use the process->address to kill the process. The inconsistency
can cause simulation to kill a protected process that is not killable and leads
to simulation failure.
2020-03-10 21:07:16 -07:00
negoyal cd949eca71 Merge branch 'master' into fdb_cache_subfeature2 2020-02-26 11:22:08 -08:00
negoyal 746523b090 Yet more compilation fixes. 2020-02-24 17:47:33 -08:00
Evan Tschannen 96258b9809 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/genericactors.actor.cpp
#	flow/serialize.h
2020-02-21 19:09:16 -08:00
Evan Tschannen f04e311a1e Merge commit 'b46d6e25e24993ab5a5f04091fd3235050b7cd09' into feature-boost-ssl
# Conflicts:
#	fdbserver/SimulatedCluster.actor.cpp
#	flow/Net2.actor.cpp
2020-02-20 17:36:38 -08:00
Evan Tschannen e06c3e2eb7 fix: checkForExcludedServer needs to check both the tls and non-tls address 2020-02-19 15:10:54 -08:00
Evan Tschannen 663d176fdb fix: coordinators auto could added 0.0.0.0:0 as a coordinator 2020-02-14 16:50:55 -08:00
negoyal 28768715f2 Custom bookkeeping allocator for PTree. Simple Cache workload. 2020-02-14 11:42:47 -08:00
Evan Tschannen 96eec756b3 more simulation fixes 2020-02-12 15:12:43 -08:00
Evan Tschannen 38a5511b96 additional simulation fixes 2020-02-11 15:52:06 -08:00
Andrew Noyes 7b5de42d43 Address review comments 2020-02-11 10:40:09 -08:00
negoyal 85cc35e81e Merge branch 'master' into HEAD 2020-02-05 14:59:55 -08:00
Andrew Noyes 90c1b2df88 Don't include header 2020-02-05 09:57:18 -08:00
mpilman 100402aadf Don't call operator explicitely 2020-02-04 11:03:43 -08:00
mpilman 52ca752dd3 Merge remote-tracking branch 'origin/features/icc' into features/icc 2020-02-04 10:29:49 -08:00
mpilman d09e07f1f5 Merge remote-tracking branch 'upstream/master' into features/icc 2020-02-04 10:26:18 -08:00
Andrew Noyes a043646d1d Don't read in init database transaction 2020-02-03 15:32:01 -08:00
Andrew Noyes 61a727d701 Allow new databases to be configured as locked 2020-02-03 15:32:01 -08:00
mpilman 1bc84cacbe Also write cacheKeys 2020-01-24 14:27:06 -08:00
mpilman ccb64e234c minor fix in changeCachedRange 2020-01-24 14:20:11 -08:00
mpilman 51717c970d Fixed management api 2020-01-24 11:00:50 -08:00
mengranwo 227edd4248 change memory storage engine name from memory-radixtree to memory-radixtree-beta 2020-01-15 13:49:45 -08:00
mengranwo f597aa7e18 WIP : deployable/stable version since Nov 3. Start rebase to master branch 2020-01-15 13:49:45 -08:00
mpilman 9c641c7a8e First part of client handling 2020-01-07 11:42:57 -08:00
Jon Fu 9db95bd976 initial commit to allow re-inclusion of servers marked as failed 2019-10-23 11:05:48 -07:00
Jon Fu d2b6626d5c Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed 2019-10-21 13:47:06 -07:00
Evan Tschannen 688940b685 merge 6.2 into master 2019-10-21 11:43:46 -07:00
Evan Tschannen 35e816e9ad added the ability to configure satellite_logs by satellite location, this will overwrite the region configure if both are present 2019-10-14 18:30:15 -07:00
Jon Fu d96a7b2c69 Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed 2019-10-03 09:47:45 -07:00
Meng Xu d0147e5e5d Merge branch 'release-6.2' into mengxu/merge-release620-to-master-v3
Resolved Conflicts:
	documentation/sphinx/source/release-notes.rst
	fdbserver/DataDistribution.actor.cpp
	versions.target
2019-10-02 13:22:56 -07:00
A.J. Beamon 1d63ba6980 Use immediate priority for coordinator changes 2019-10-01 08:36:37 -07:00
Evan Tschannen 324d0bd3b0 Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-cleanup-mutations 2019-09-27 19:15:14 -07:00
Evan Tschannen 3cc5d484a5 the include and exclude commands do not need to set the moveKeysLockOwnerKey, which will kill the data distribution algorithm 2019-09-27 18:33:56 -07:00
Jon Fu 450a09e117 Code Review Changes 2019-09-24 15:48:50 -07:00
Jon Fu 471e283128 Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed 2019-09-18 11:49:07 -07:00
sramamoorthy a4d38f1158 Fix #2057 snapshot cli to print UID in failure too 2019-09-17 05:18:28 -07:00
Jingyu Zhou d9fb199486 Remove debug messages from checkDatabaseLock() 2019-09-12 22:11:25 -07:00
Evan Tschannen 8fbd90e2f6
Merge pull request #1985 from xumengpanda/mengxu/storage-engine-switch-PR-v2
Graceful storage engine migration
2019-09-09 13:51:53 -07:00
Meng Xu c2355f721e Merge branch 'master' into mengxu/performant-restore-PR 2019-09-04 17:11:42 -07:00
Meng Xu d160810662 FastRestore:Resolve review comments 2019-09-04 16:48:43 -07:00
Meng Xu bd80a67d46 Merge branch 'master' into mengxu/storage-engine-switch-PR-v2 2019-09-03 14:11:33 -07:00
Evan Tschannen dc1d055b27
Merge pull request #2042 from senthil-ram/snap_cli_fix
fix fdbcli --exec 'snapshot create.sh' failure
2019-08-30 13:40:38 -07:00
sramamoorthy b3277f2982 Fix #2009 posix compliant args for snapshot binary 2019-08-30 12:54:09 -07:00
A.J. Beamon 3f9e392668
Merge pull request #2014 from etschannen/feature-fdbcli-sleep
Added a sleep command to fdbcli
2019-08-30 11:22:13 -07:00
Evan Tschannen f3bc7e0abd do not duplicate data distribution disabled fields in status
fixed a few bugs related to the existing data distribution disabled fields in status
2019-08-29 18:41:34 -07:00
Jon Fu 807b02551e updated help message and changed existing workload to use mark as failed feature 2019-08-27 14:39:43 -07:00
Jon Fu c908c6c1db added command to fdbcli and changes to SystemData and ManagementAPI 2019-08-27 14:39:43 -07:00
Evan Tschannen 0b0c9fe0ff data distribution status was combined into regular status 2019-08-21 14:44:15 -07:00
A.J. Beamon 7953545331 Fix an unknown_error when the file passed to fileconfigure doesn't contain a valid object (e.g. if you omit the enclosing {} of your object).
Fix an internal error when configuring regions with some storage servers that don't have a datacenter set.
2019-08-19 11:28:15 -07:00
Evan Tschannen 2a436d5f6f fix: do not block fdbcli from starting if DataDistributionStatus is not available 2019-08-16 18:15:02 -07:00
Meng Xu e6284684f0 StorageEngineSwitch:Always remove wrong storeType SS
In the old logic of switching storage engines, it marks a storage server
with wrong store type as undesired even though this can lead to no healthy team.

In the first version of the new storage engine switch, we mimic the same logic
of the old version.
2019-08-13 14:59:46 -07:00
Meng Xu a588710376 StorageEngineSwitch:Graceful switch
When fdbcli change storeType for storage engines,
we switch the store type of storage servers one by one gracefully.
This avoids recruiting multiple storage servers on the same process,
which can cause OOM error.
2019-08-12 17:37:52 -07:00
Meng Xu 3b54363780 FastRestore:Apply Clang-format 2019-08-01 18:09:12 -07:00
Meng Xu 7ccaeddf05 Merge branch 'master' into mengxu/performant-restore-PR 2019-08-01 13:23:17 -07:00
Xin Dong 5d20364423 Address review comments 2019-07-30 22:24:30 -07:00
Xin Dong 1922c39377 Resolve review comments. 100K run shows one suspecious ASSERT_WE_THINK failure which I think could be a race. 2019-07-30 22:24:30 -07:00
Xin Dong ae11efcb0a Made following changes:
- Make sure the disabled data distribution won't be accidentally enabled by the 'maintenance' command
- Make sure the status json reflects the status of DD accordingly
- Make sure the CLI can play with the new DD states correctly, i.e. print out warns when necessary
2019-07-30 22:20:45 -07:00
sramamoorthy 49eaa31984 Add a trace event for snap create failure 2019-07-29 20:28:22 -07:00
sramamoorthy 5a56f6b456 minor snap create client improvement and bug fixes 2019-07-29 20:28:22 -07:00
Meng Xu 1706aaf199 Merge branch 'master' into mengxu/performant-restore-PR
Fix conflict in TlogServer.actor.cpp by accepting master changes
2019-07-26 11:46:27 -07:00
sramamoorthy 9afd162e2f remove snap v1 related code 2019-07-25 17:29:31 -07:00
Meng Xu 45083edf74 Merge branch 'master' into mengxu/performant-restore-PR
Fix conflicts as well.
2019-07-25 10:46:11 -07:00
sramamoorthy a2f2ad96ff code review comments and merge to master changes 2019-07-24 15:36:28 -07:00
sramamoorthy 209448807d snap v2: fdbclient related changes 2019-07-24 15:36:28 -07:00
Evan Tschannen 83f4b8ebb1
Merge pull request #1866 from senthil-ram/setDDBugFix
setDDMode should set moveKeysLockWriteKey
2019-07-24 15:35:18 -07:00
sramamoorthy 0962641540 setDDMode should set moveKeysLockWriteKey
After takeMoveKeysLock notes down the owner and the
moveKeysLockWriteKey value, it monitors the above two in
pollMoveKeysLock and checks if anything is changed, but
setDDMode was not setting the moveKeysLockWriteKey and
so a sequence like disable, enable and disable would not
really disable DD.
2019-07-19 11:13:29 -07:00
Markus Pilman b242760adf
Use functional cast instead of explicit operator call 2019-07-19 09:58:40 -07:00
Vishesh Yadav d9a8657096 fdbcli: Add `no_wait` option in `exclude` command to avoid blocking
RESOLVES #1840
2019-07-18 13:07:31 -07:00
Alex Miller 7a500cd37f A giant translation of TaskFooPriority -> TaskPriority::Foo
This is so that APIs that take priorities don't take ints, which are
common and easy to accidentally pass the wrong thing.
2019-06-25 02:47:35 -07:00
mpilman 844dd60202 FDB compiling with intel compiler 2019-06-20 09:29:01 -07:00
Meng Xu 477fd152c0 FastRestore:Refactor code
1) Use the runRYWTransaction for simple DB access
2) Replace some printf with TraceEvent
3) Remove printf not used in debugging
4) Avoid wait inside the condition in loop-choose-when for
   the core routine of restore worker, loader and applier.
5) Rename Restore.actor.cpp to RestoreWorker.actor.cpp since
   the file only has functionalities related to restore worker.

Passed correctness test
2019-06-04 11:22:47 -07:00
sramamoorthy 4bcb590f12 g_random -> deterministicRandom() 2019-05-28 22:07:46 -07:00
sramamoorthy ec7834e2f7 code re-orgnaization and address comments 2019-05-28 22:07:46 -07:00
sramamoorthy 61e93a9304 Address review comments and minor fixes 2019-05-28 22:07:46 -07:00
sramamoorthy 69edefe68b Snapshot based backup and resotre implementation 2019-05-28 22:07:46 -07:00
A.J. Beamon 603721e125 Merge branch 'master' into thread-safe-random-number-generation
# Conflicts:
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/genericactors.actor.cpp
#	fdbrpc/sim2.actor.cpp
#	fdbserver/DiskQueue.actor.cpp
#	fdbserver/workloads/BulkSetup.actor.h
#	flow/ActorCollection.actor.cpp
#	flow/Net2.actor.cpp
#	flow/Trace.cpp
#	flow/flow.cpp
2019-05-23 08:35:47 -07:00
Evan Tschannen f4fbaac6b0 Merge branch 'release-6.1'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	versions.target
2019-05-19 10:27:59 -07:00
Evan Tschannen 4166ecdfe7 Increase the configure timeout to 60 seconds, to avoid spurious configuration failures 2019-05-16 18:03:16 -07:00
A.J. Beamon 5f55f3f613 Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used. 2019-05-10 14:01:52 -07:00
Austin Seipp b5cbffc1b8 fdbclient: fix some print/scan format warnings
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-05-06 13:35:29 -07:00
Evan Tschannen 2d5043c665 Merge branch 'release-6.1'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	versions.target
2019-04-30 18:27:04 -07:00
Alex Miller 6124a7e145 Update fdbclient/ManagementAPI.actor.cpp 2019-04-30 16:30:19 -07:00
Evan Tschannen b2d19eebc4 fdbcli would return success even when configure failed for a variety of error types
the existing configure safety check would fail when attempting to change between three_datacenter and region configuration
2019-04-30 16:30:19 -07:00
Meng Xu 529ce66b6c Merge branch 'apple/master' into mengxu/performant-restore-PR 2019-04-18 18:02:45 -07:00
Evan Tschannen cd5c9d91fa
Merge pull request #1443 from etschannen/master
Merge 6.1 into master
2019-04-10 17:43:07 -07:00
Evan Tschannen 6220a5ce0f
Merge pull request #1370 from jzhou77/fix-unreferenced
Remove unused functions
2019-04-09 11:49:45 -07:00
Evan Tschannen 3356ac27bf added three_data_hall_fallback configuration 2019-04-07 22:58:18 -07:00
mpilman 1c16f87a4e Remove trace-calls to printable (in non-workloads) 2019-04-05 13:12:19 -07:00
Meng Xu c4a8a80d6f Merge branch 'apple/master' into mengxu/performant-restore-PR 2019-04-04 22:51:00 -07:00
Evan Tschannen a38c396283 made all maintenance transactions lock aware 2019-04-02 14:27:48 -07:00
Evan Tschannen 781cf9b5a0 added the ability to make a zoneId for maintenance in fdbcli 2019-04-01 17:55:13 -07:00
Jingyu Zhou 47b4b82628
Merge branch 'master' into fix-unreferenced 2019-04-01 14:07:19 -07:00
Jingyu Zhou f7f8ddd894 Fix warnings on unused variables
Found by -Wunused-variable flag.
2019-04-01 14:00:20 -07:00
Meng Xu 70d7c289f4 Merge branch 'master' into mengxu/restore/parallel-v7 2019-03-30 22:13:10 -07:00
Evan Tschannen b6008558d3 renamed BinaryWriter.toStringRef() to .toValue(), because the function now returns a Standalone<StringRef>()
eliminated an unnecessary copy from the proxy commit path
eliminated an unnecessary copy from buffered peek cursor
2019-03-28 11:52:50 -07:00
Evan Tschannen f9aad46573 made use_provisional_proxies a transaction option 2019-03-19 18:44:37 -07:00
Evan Tschannen 5b9c45ea0b clients do not attempt to connect to provisional proxies 2019-03-19 13:37:50 -07:00
Evan Tschannen eb54a700ba changed the old memory configuration to memory-1 2019-03-18 15:10:04 -07:00
Evan Tschannen a372c7cf18 configure memory now selects the ssd engine for transaction log spilling. Transaction log spilling is only used when the transaction logs cannot keep all of the unpopped mutations it has in memory. If we are only using this data structure because we do not have enough memory, it is much less safe to use the memory storage engine for this purpose. 2019-03-16 22:48:24 -07:00