Commit Graph

208 Commits

Author SHA1 Message Date
Alex Miller bc8e6acbe8 Fix the other half of simulation requiring a TLS Plugin.
This commit:
1. Restores --tls_plugin as a way to provide the path to the TLS plugin when running in simulation.
2. Removes the TLS Plugin as being required for 5% of tests.
3. Standardizes on 'sslEnabled' as a variable name.

And is a fix/improvement upon commit f7733d1b.

(1) previously didn't work, because we would create multiple new TLSOptions
instances and run init_plugin multiple times.  Only the first call would use
the argument specified on the command line.  To fix this, the TLSOptions
derived from the command line is threaded through all the simulation code that
needs it.

(2) was an oversight in f7733d1b, which didn't actually make "should we be TLS"
dependant on if the TLS plugin was available or not.

(3) is just nice for trying to grep around in the codebase.
2018-04-30 18:26:29 -07:00
Evan Tschannen 883f2318a0 test fearless configurations 2018-04-30 13:17:29 -07:00
Alec Grieser 69e831d522
Merge remote-tracking branch 'upstream/release-5.2' into merge-release-5.2 2018-04-28 17:44:52 -07:00
Yichi Chiang c721ab6854 Fix review comments 2018-04-27 13:54:34 -07:00
Yichi Chiang 6bddf8aefa Upgrade DR from 5.1 to 5.2 2018-04-26 17:24:40 -07:00
Alex Miller f7733d1bd0 Do not require the TLS Plugin for simulation.
It appears that explicit calls to TLS-related things had snuck in over time,
which meant that simulation runs that weren't even configured to use SSL still
wanted and required the TLS plugin.

This commit instead threads through the understanding of if any TLS-related
options were provided, and if not, then don't call anything TLS-related so that
we don't require the TLS plugin.

Hopefully this makes life easier for the opensource folk. :)
2018-04-24 16:53:30 -07:00
Evan Tschannen d46d5487bd Merge branch 'release-5.2' 2018-04-18 20:46:03 -07:00
Evan Tschannen 57d650062a merge 5.1 into 5.2 2018-04-18 20:44:31 -07:00
Evan Tschannen 224621be04 fix: extraDB==0 must leave g_simulator.extraDB as null, so that non-DR tests do not attempt to use a DR database 2018-04-18 19:34:35 -07:00
Evan Tschannen c6229e443c fix: do not use resolution class when using regions 2018-04-11 21:22:53 -07:00
Evan Tschannen 19762b847d Merge branch 'release-5.2'
# Conflicts:
#	fdbserver/DatabaseConfiguration.cpp
#	fdbserver/SimulatedCluster.actor.cpp
2018-04-10 17:02:43 -07:00
Evan Tschannen c1ba16b3c8 Merge branch 'release-5.1' into release-5.2
# Conflicts:
#	bindings/java/src/test/com/apple/foundationdb/test/AbstractTester.java
#	bindings/java/src/test/com/apple/foundationdb/test/VersionstampSmokeTest.java
#	bindings/nodejs/lib/fdb.js
#	bindings/nodejs/src/Version.h
#	bindings/nodejs/tests/tuple_test.js
2018-04-10 16:50:47 -07:00
Evan Tschannen b0a88001cc
Merge pull request #132 from yichic/support-dr-upgrade-test
Support DR upgrade test
2018-04-10 16:30:19 -07:00
Yichi Chiang d0230d4d13 Support DR upgrade test in 5.1 2018-04-10 15:19:53 -07:00
Alex Miller 0136a01c18 Fix "Not enough physical servers available" error due to incorrect server calculation. 2018-04-05 15:13:21 -07:00
Evan Tschannen 4fb2b99341 fix: using only one region still means we need 3 machines per datacenter, the other machines in the other datacenters just won’t be used 2018-03-30 19:26:22 -07:00
Evan Tschannen b36e08f08f first version of non-copying recovery. Upgrades are broken, and it has not been tested using fearless configurations yet 2018-03-29 15:12:38 -07:00
Evan Tschannen 82ed956c65 renamed the multi_dc configuration to three_datacenter. The old three_datacenter configuration was not a useful configuration. 2018-03-26 18:31:26 -07:00
Evan Tschannen 5db52ab081
Merge pull request #87 from etschannen/feature-remote-logs
Feature remote logs
2018-03-23 12:55:17 -07:00
Evan Tschannen 7c48e1d31c
Update SimulatedCluster.actor.cpp 2018-03-23 12:54:44 -07:00
yichic ede5cab192
Merge pull request #89 from yichic/share-log-mutations-5.2
Share log mutations 5.2
2018-03-19 12:01:26 -07:00
Yichi Chiang d6559b144f Share log mutations between backups and DRs which have the same backup range 2018-03-19 11:32:50 -07:00
Yichi Chiang 26b93ff920 Share log mutations between backups and DRs which have the same backup range 2018-03-16 18:09:23 -07:00
Evan Tschannen a42205eb8e test running with only one region 2018-03-15 15:40:58 -07:00
Evan Tschannen 65b532658f added support for single region configurations 2018-03-15 10:59:30 -07:00
A.J. Beamon f2c804e14f Reverting changes from merge of master into release-5.2 (b25810711c). Note that we never intend to release master into release-5.2, but if we did we would need to revert this commit. 2018-03-06 10:15:04 -08:00
Evan Tschannen 1194e3a361 added region-based configuration to support a large variety of fearless setups. Currently only 1 primary 1 remote setups are allowed. 2018-03-05 19:27:46 -08:00
Evan Tschannen 470f5c01f3 changed remoteDcId to a vector of ids, to support future configurations where there are multiple remote databases 2018-02-26 17:09:09 -08:00
Evan Tschannen a67296b373 do not test fearless configurations to merge with master 2018-02-26 13:31:06 -08:00
Evan Tschannen 8e966fdf9c simulated cluster tests all configurations. Still needs to randomize the remote and satellite replication, along with them number of remote tlogs, log routers, and satellite tlogs 2018-02-26 13:15:44 -08:00
Evan Tschannen 37a6a81634 Merge commit '7f6fc3e039c911cd84b8540f7f799fc38a1c1822' into feature-remote-logs
# Conflicts:
#	fdbserver/workloads/RestartRecovery.actor.cpp
2018-02-23 12:33:28 -08:00
Alec Grieser 0bae9880f1 remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py 2018-02-21 10:25:11 -08:00
Evan Tschannen 1b5628d2c5 testing a single configured fearless setup in simulated cluster
consolidated simulation connection disablers into one call in the tester
automatically reconfigure from a fearless setup in simulation
2018-02-18 12:59:43 -08:00
Evan Tschannen cb25564d38 simulated cluster supports fearless configurations
removed unused simulation variables
run the simulation with only 1 coordinator most of the time, since we protect the coordinator from being killed, and protecting too many things is bad for simulation
2018-02-15 18:32:39 -08:00
Evan Tschannen ad19d3926b fix: make sure there are enough machines in each dc to support triple replication for the configure workload 2018-02-14 17:06:22 -08:00
Evan Tschannen a52ea4eb78 restored 5.1 functionality of simulated cluster. Will test assigned primary and remote data centers. Does not test remote replication or satellite logs 2018-02-10 13:27:51 -08:00
Evan Tschannen ebd94bb654 removed a separately configurable storage team size for the remote data center, because it did not make sense
fix: the master did not monitor for the failure of remote logs
stop merge attempts when a data center is failed
fixed a variety of other problems with data distribution when a data center is failed
2018-02-02 11:46:04 -08:00
Evan Tschannen 5ac4f73978 Merge branch 'release-5.1' into feature-remote-logs
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
#	fdbrpc/Locality.h
#	fdbrpc/simulator.h
#	fdbserver/ApplyMetadataMutation.h
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/LogSystemPeekCursor.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/SimulatedCluster.actor.cpp
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/TagPartitionedLogSystem.actor.cpp
#	fdbserver/WorkerInterface.h
#	fdbserver/masterserver.actor.cpp
#	flow/Net2.actor.cpp
#	tests/fast/SidebandWithStatus.txt
#	tests/rare/LargeApiCorrectnessStatus.txt
#	tests/slow/DDBalanceAndRemoveStatus.txt
2018-01-05 11:33:42 -08:00
Evan Tschannen 8c51bc4ac4 fixed low latency tests in a way that gives us better test coverage 2017-11-28 18:20:29 -08:00
Alec Grieser 5a4a5985fd Merge branch 'release-5.0' 2017-10-30 08:31:23 -07:00
Alec Grieser 87321f5017 Merge branch 'release-4.6' into release-5.0 2017-10-30 08:31:01 -07:00
Evan Tschannen 7a36fd2134 disabled a variety of simulation tests to get correctness clean 2017-10-19 15:49:54 -07:00
Alec Grieser e0be1ef1e0 Merge branch 'release-5.0' 2017-10-16 10:08:11 -07:00
Alec Grieser 432726ba2d Merge branch 'release-4.6' into release-5.0 2017-10-16 09:54:21 -07:00
Alex Miller 567d663afd Fix SimulationConfig never generating a custom config.
A 0 was changed to a 1 when rewriting code, and `case 0:` was never being hit. :(
Thankfully, it looks like nothing was broken by this in the meantime.
2017-09-18 17:29:36 -07:00
Evan Tschannen 34f987f56d added a test in simulation which ensures that a recovery after a single failure takes less than 15 seconds 2017-09-15 17:55:01 -07:00
A.J. Beamon 9a0a3b6329 Merge commit '66528becb82d826e81fa644bb378212584ab580e' 2017-08-28 16:47:59 -07:00
Alvin Moore 44e0df78c5 Added support for tracking roles for simulation workers
Fixed the exclusion and inclusion address simulation API and integration within workloads
Added more information within trace events for simulation
2017-08-28 11:25:37 -07:00
Alec Grieser 300b5a17ed Merge branch 'release-5.0' 2017-08-25 18:55:33 -07:00
Evan Tschannen 272b4b984c fix: fixed a rare bug where we do not wait for a file in the process of being deleted to shutdown before rebooting a machine 2017-08-25 10:12:58 -07:00
Alex Miller 7b78035365 Have SimulationConfig wrap DatabaseConfiguration to reduce code duplication.
This effectively turns initializing SimulationConfig into the equivalent of
building a config string and calling buildConfiguration on it.
2017-08-22 10:13:57 -07:00
Alex Miller 9b25c72971 Pull database config and cluster config into one struct.
This will allow us to specify custom situations to be chosen more frequently,
and in particular control machines and processes.
2017-08-21 22:35:44 -07:00
Evan Tschannen 9fd5955e92 Merge branch 'master' into removing-old-dc-code 2017-06-26 16:27:10 -07:00
Alvin Moore 9553458b78 Updated simulation to support managing exclusion and inclusion address
Added method for identifying acceptable availability process classes
Extended cluster availability function to ensure coordinators can be auto configured
Fixed availability function to allow protected processes to be considered as dead if not available
Added debug trace events for providing machine state when considering availability
Added trace event for protected coordinators
2017-06-19 16:48:15 -07:00
Evan Tschannen 766dc23e26 fix: do not use TLS in protectedAddresses 2017-06-02 13:52:21 -07:00
Stephen Atherton 98604d33a0 Merge branch 'fix-io-timeout-handling'
# Conflicts:
#	fdbrpc/AsyncFileKAIO.actor.h
#	fdbrpc/sim2.actor.cpp
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/optimisttest.actor.cpp
#	fdbserver/worker.actor.cpp
#	fdbserver/workloads/MachineAttrition.actor.cpp
#	tests/fast/SidebandWithStatus.txt
#	tests/rare/LargeApiCorrectnessStatus.txt
#	tests/slow/DDBalanceAndRemoveStatus.txt
2017-05-26 18:43:08 -07:00
Alvin Moore 0b9ed67e12 Fixed support for RemoveServers Workload
Added availability functions to simulation
2017-05-26 14:20:11 -07:00
FDB Dev Team a674cb4ef4 Initial repository commit 2017-05-25 13:48:44 -07:00