eb7ad07df0
Final cleanups.
2020-07-12 19:53:44 -07:00
2876763623
Add errors if unknown params are used in a global or test section.
2020-07-12 19:41:09 -07:00
0131f0dc48
Make simulation able to load and run .toml files
2020-07-12 14:47:39 -07:00
464e9332e4
Refactor testspec parsing.
...
This separates the file format and how to read it from how to apply a key/value
to a TestSpec. This will allow us to reuse the same code when implementing a
TOML parser later.
2020-07-12 14:47:39 -07:00
17570b5b10
Make the testspec more restrictive in terms of what can be set where.
...
Testspec is currently very permissive in very misleading ways. In particular,
the tester parser itself will swallow K=V settings and apply them at the test
level, which breaks how a person would expect the scoping to work. Other
settings apply to the entire simulation run globally, but appear to be workload
specific. Even further, others affect simulation cluster creation or test
harness behavior, but can again be set anywhere in a testspec.
This changes testspec parsing to error if a setting that applies globally is
anywhere but the top of the file, or if a setting that applies test-wide is
applied to a workload instead of a test.
2020-07-06 02:03:30 -07:00
5f9b127e56
Emit traces regularly about role assignment
...
We are currently emitting Role transition traces when a role starts and
when it ends. While this is useful for debugging, it doesn't work well
with tools that inject data and might potentially miss some trace lines.
We do decorate each trace lines with the roles assigned to that
particular process, however, this is not sufficient for tools that can
make use of the UID -> Role mapping
2020-05-08 16:27:57 -07:00
20d6f2a3c3
Merge remote-tracking branch 'origin/master' into bugfixes/fix-clang10-compiler-warnings
2020-05-06 15:50:39 -07:00
f967209ea0
Fix compiler warnings with clang
2020-04-22 14:14:56 -07:00
1476057996
properly cache serialization of serverDBInfo
2020-04-11 19:30:05 -07:00
07cc0a8d74
code cleanup
2020-04-10 17:02:11 -07:00
2a1bd97120
fix compilation errors
2020-04-06 20:58:43 -07:00
0b0414fb94
Addressded review comments. Change the issue reporting from 'ITraceLogWriter' to be a more generic way.
2020-02-25 15:37:53 -08:00
034dfe5e42
Now the inability to flush trace logs will be reported to both 'stderr' and also the status json object.
...
- Since the first flush failure, if the accumulated consecutive failure count exceeds the value defined in knobs, it will trigger the current worker process to report this issue via the 'GetServerDBInfo' interface of the cluster controler
- A successful flush will reset the accumulated counter.
Notice that the current solution does not take the time into consideration. The assumption is that flush failures tend to only happen in a clustered manner. The intermittent, but short, periods of flush failures are not considered as a problem since the memory pressure built by them should be negligible.
2020-02-25 15:37:32 -08:00
96258b9809
Merge branch 'release-6.2'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbcli/fdbcli.actor.cpp
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/ClusterController.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/DataDistribution.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/KeyValueStoreMemory.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/QuietDatabase.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/StorageMetrics.actor.h
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/KVStoreTest.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/genericactors.actor.cpp
# flow/serialize.h
2020-02-21 19:09:16 -08:00
17660fb18d
Fix simulation test
2020-02-11 13:49:19 -08:00
3b57bf1781
Merge branch 'master' into mengxu/fast-restore-agent-PR
2020-02-03 17:23:54 -08:00
559b95c61a
FastRestore:RestoreRole:Mimic how fdbd starts
2020-02-01 10:23:48 -08:00
6e6cfaff16
Cleanup old Failure Monitoring code
2020-01-07 15:53:32 -08:00
8f84fbc4b9
Only print 'waiting for DD to end...' if test actually waits
2019-11-03 16:13:32 -08:00
58b984b846
Merge pull request #2047 from tclinken/lock-aware-db-ping
...
Use lock aware transaction for pingDatabase
2019-10-31 10:24:20 -07:00
0b785e5c1c
DD:getTeam may fail to get a team when it can
...
Due to randomness, when unhealthy teams are majority while there still
exists healthy teams, getTeam function may be unlucky to find
any feasible (ok) team, which leads to BestTeamStuck situation.
This commit increases the tries from 10 to 20.
A long-term solution may first find all feasible teams and choose a random
one from them. Since This can affect the statistics of which team is picked.
So it is not included in this commit.
Non-functional change: This commit removes unneeded printf introduced by
fast restore PR 1404.
2019-09-07 20:08:58 -07:00
2f92cf8c96
Use lock aware transaction for pingDatabase
2019-09-07 12:25:44 -07:00
c2355f721e
Merge branch 'master' into mengxu/performant-restore-PR
2019-09-04 17:11:42 -07:00
6aa0ada7b1
Replace scalar root types with proper messages
2019-08-28 14:40:50 -07:00
3b54363780
FastRestore:Apply Clang-format
2019-08-01 18:09:12 -07:00
7ccaeddf05
Merge branch 'master' into mengxu/performant-restore-PR
2019-08-01 13:23:17 -07:00
90e3b50213
Merge branch 'master' into feature-coordinator-connection
...
# Conflicts:
# fdbclient/DatabaseContext.h
# fdbclient/NativeAPI.actor.cpp
# fdbclient/NativeAPI.actor.h
# fdbserver/workloads/KillRegion.actor.cpp
2019-07-26 15:05:02 -07:00
45083edf74
Merge branch 'master' into mengxu/performant-restore-PR
...
Fix conflicts as well.
2019-07-25 10:46:11 -07:00
c18558cf55
enable DD mode in restore based on test spec
2019-07-24 15:36:28 -07:00
62c14dae72
disable dd during snap and enable in restore
2019-07-24 15:36:28 -07:00
4a866290b7
Clients keep a persistent connection open with coordinators to get updates to the list of proxies
...
Status still needs to be updated with client information with information from the coordinators
2019-07-23 19:22:44 -07:00
a5a6f8431c
Add a random UID to TransactionMetrics in case a client opens multiple connections and also a field to indicate whether the connection is internal. Convert some of the metrics to our Counter object instead of running totals.
2019-07-08 14:01:04 -07:00
603721e125
Merge branch 'master' into thread-safe-random-number-generation
...
# Conflicts:
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/genericactors.actor.cpp
# fdbrpc/sim2.actor.cpp
# fdbserver/DiskQueue.actor.cpp
# fdbserver/workloads/BulkSetup.actor.h
# flow/ActorCollection.actor.cpp
# flow/Net2.actor.cpp
# flow/Trace.cpp
# flow/flow.cpp
2019-05-23 08:35:47 -07:00
f6fbad5061
Fix memory bug
2019-05-13 14:15:23 -07:00
5f55f3f613
Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used.
2019-05-10 14:01:52 -07:00
bf378952cb
fdbserver: fix some print/scan format warnings
...
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-05-06 13:35:29 -07:00
529ce66b6c
Merge branch 'apple/master' into mengxu/performant-restore-PR
2019-04-18 18:02:45 -07:00
ef04471a66
Fix more unused-variable warnings
2019-04-17 16:04:10 -07:00
1c16f87a4e
Remove trace-calls to printable (in non-workloads)
2019-04-05 13:12:19 -07:00
70d7c289f4
Merge branch 'master' into mengxu/restore/parallel-v7
2019-03-30 22:13:10 -07:00
0bbdc15f71
Multi-test processes waits until a timeout if any of the tester processes restarts. Use getReplyUnlessFailedFor instead of getReply to detect the restarts and fail quickly instead of waiting for a timeout which is usually large.
2019-03-28 17:05:30 -07:00
710a64dc4e
replaced std::pair<WorkerInterface,ProcessClass> with a struct named WorkerDetails
2019-03-08 11:25:07 -05:00
e2bcecb08f
Merge branch 'master' into ratekeeper-batch-priority-limits
2019-02-28 12:52:44 -08:00
fafb66b0a8
Merge pull request #1126 from bnamasivayam/ratelimit-consistencycheck
...
Dynamically rate limit consistency check.
2019-02-27 14:43:09 -08:00
a051055caf
Initial implementation of adding separate limits for batch priority in ratekeeper
2019-02-27 10:31:56 -08:00
999ea09bfd
Use correct fwd decls in TesterInterface
...
Also TesterInterface.h -> TesterInterface.actor.h
2019-02-19 15:16:59 -08:00
699216f713
Use fwd decls in workloads
...
Also workloads.h -> workloads.actor.h
2019-02-19 15:16:59 -08:00
3f0fd2a20c
Use fwd decls in WorkerInterface
...
Also WorkerInterface.h -> WorkerInterface.actor.h
2019-02-19 15:16:59 -08:00
0bb60e5a3b
Use proper fwd decl in NativeAPI
...
Also NativeAPI.h -> NativeAPI.actor.h
2019-02-19 15:16:59 -08:00
3cb2391b58
use proper fwd declarations in ManagementAPI
...
Also ManagementAPI.h -> ManagementAPI.actor.h
2019-02-19 15:16:59 -08:00
Alex Miller