0b1984978a
Small code refactoring.
2019-04-21 10:41:07 -07:00
ec1bc5cfca
Add LogSystemType enum
2019-04-21 10:41:07 -07:00
6220a5ce0f
Merge pull request #1370 from jzhou77/fix-unreferenced
...
Remove unused functions
2019-04-09 11:49:45 -07:00
1c16f87a4e
Remove trace-calls to printable (in non-workloads)
2019-04-05 13:12:19 -07:00
47b4b82628
Merge branch 'master' into fix-unreferenced
2019-04-01 14:07:19 -07:00
e7ad39246c
Fix typo
2019-03-29 20:16:26 -07:00
a44ffd851e
fix: the shared tlog could fail to update a stopped tlog’s queueCommitVersion to version if a second tlog registered before it could issue the first commit for the tlog
2019-03-29 20:11:30 -07:00
b6008558d3
renamed BinaryWriter.toStringRef() to .toValue(), because the function now returns a Standalone<StringRef>()
...
eliminated an unnecessary copy from the proxy commit path
eliminated an unnecessary copy from buffered peek cursor
2019-03-28 11:52:50 -07:00
a55f06e082
Remove unused functions
...
Found with -Wunused-function flag.
2019-03-27 15:45:28 -07:00
c705a1af74
fix: make sure recoveryLocation is always a valid page
2019-03-20 19:33:09 -07:00
1c6ad6d307
fix: change the location where stopped is checked, because a yield could cause cause stopped to be set after the existing check
2019-03-20 19:33:09 -07:00
b11ecb3210
Remove random bits of code that were either unneeded or leftover from debugging.
2019-03-18 15:47:20 -07:00
37ea71b117
Implement limiting how many bytes recovery will read.
...
This time, track what location in the DiskQueue has been spilled in
persistent state, and then feed it back into the disk queue before
recovery.
This also introduces an ASSERT that recovery only reads exactly the
bytes that it needs to have in memory.
2019-03-18 15:09:43 -07:00
29ab7370cd
Clear versionLocation when spilling, and pop DQ separately.
...
Popping the disk queue now requires potentially recovering the location
to which we can pop from the spilled data itself, and for each tag we
must maintain the first location with relevant data.
The previous queue we had to represent the ordering, queueOrder, was
used by spilling, and popped when a TLog had been spilled. This means
that as soon as a TLog has been fully spilled, we have no idea how it
relates in order to other fully spilled TLogs.
Instead, use queueOrder to keep track of all the TLog UIDs until they're
removed, and use spillOrder to keep track of the order only for
spilling.
2019-03-18 15:09:22 -07:00
7f5bc2981f
Checksum DiskQueue pages on read, but at a lower priority.
...
If a server has its data spilled, then it's behind the 5s window.
Feeding it data is less important than committing, so we can hide the
extra CPU usage from checksumming the read amplified disk queue pages.
2019-03-15 21:01:19 -07:00
ee4721a63f
Make checking or ignoring checksums part of the IDiskQueue::read API.
2019-03-15 21:01:18 -07:00
81c59e88a8
Persist the protocol version of a TLog instance when it is created.
...
This allows us to do easy upgrades of SpilledData in the future, if the
need arises, because we then have a protocol version to compare against.
2019-03-15 21:01:17 -07:00
686b097397
Remove verification code from DiskQueue and TLogServer.
2019-03-15 21:01:15 -07:00
77f596743f
Bump persistFormat in TLogServer to differ from OldTLogServer*
...
Though this format is being deprecated in favor of an eventual plumbing
through of TLogVersion, we should probably bump it anyway.
And also remove the fallback to OldTLogServer code. It should never be
executed, as OldTLogServer_6_0 is entirely relied upon to execute
OldTLogServer_4_6.
2019-03-15 21:01:13 -07:00
4f98634f59
Add LogId to all TLog TraceEvents that have it.
2019-03-15 21:01:12 -07:00
5873705228
tlog commits very rarely take an additional 6 seconds
2019-03-11 12:11:17 -07:00
80c3f2f8e2
added status fields detailing which processes are degraded, and also the total number of degraded processes
2019-03-10 22:58:15 -07:00
044b6b4f8a
Merge branch 'master' into feature-degraded-tlog
...
# Conflicts:
# fdbserver/ClusterController.actor.cpp
2019-03-08 22:50:41 -05:00
53f16b5347
when a tlog queue commit takes longer than 5 seconds, its process is marked as degraded
2019-03-08 11:46:34 -05:00
c6a65389ae
Remove noexcept macro and replace with BOOST_NOEXCEPT.
...
BOOST_NOEXCEPT does what the noexcept macro was supposed to do, but in a
way that is correctly maintained over time.
2019-03-05 22:06:12 -08:00
244903a9de
Spill txsTag by value under TagMsg/ and not TagMsgRef/
...
There's not a tremendous reason as to why this matters now, but I feel
like I might regret sometime later not keeping the same schema under the
same key.
2019-03-04 01:42:39 -08:00
72c2cf11ab
Replace ResourceLimiter with FlowLock.
2019-03-04 01:42:38 -08:00
aff9ebe21a
Spill (start,length) instead of (begin,end) to save a few bytes.
2019-03-04 01:42:38 -08:00
9ef283d4e7
Implement hard limiting of memory used to serve peek requests.
2019-03-04 01:42:38 -08:00
e3506ad9af
Add a yield to parseMessagesForTag
2019-03-04 01:42:38 -08:00
742f6e1847
Solve overreading via pre-calculating tag bytes per commit
2019-03-04 01:42:38 -08:00
e7d8520c63
Batch more when spilling data.
2019-03-04 01:42:38 -08:00
04e1170c88
Spill txsTag by value
2019-03-04 01:42:38 -08:00
ba31f8f1f9
Remove all code related to writing and cleaning up old-style spilling.
2019-02-26 18:13:49 -08:00
0539c1df00
Peek from new spilled data and not old spilled data.
2019-02-26 18:13:49 -08:00
d687a4bb85
Implement proper cleanup of disk queue when spilling refs.
2019-02-26 18:00:55 -08:00
84dc41c206
Add a comment.
2019-02-26 18:00:55 -08:00
cade914645
Temporarily add verification of spilled tag data.
2019-02-26 18:00:55 -08:00
f659825575
Persist start,end of message when spilling tags.
2019-02-26 18:00:55 -08:00
8d76cbed02
Track both start and end versions in versionLocation.
2019-02-26 18:00:55 -08:00
8afb7fbb9d
Merge pull request #1160 from alexmiller-apple/tstlog-fork
...
Spill-By-Reference TLog Part 2: New and Old TLogServers co-exist harmoniously
2019-02-26 18:00:04 -08:00
2dc57568cb
Change many things about log_version.
...
* log_version in the database (`/conf/log_version`) is now a hint that gets
rounded to the nearest supported version.
* fdbcli and FDB enforce that only a valid log_version can be configured to
* TLogVersion is persisted in CoreTLogSet (and LogSet and TLogSet)
* Some comments here and there
* Add an assert on filename length to make sure KV-pairs in filename
don't exceed a maximum length.
2019-02-26 16:47:04 -08:00
b8910ba7cd
Merge branch 'master' into feature-fix-force-recovery
...
# Conflicts:
# fdbclient/ManagementAPI.actor.h
# fdbserver/DataDistribution.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/KillRegion.actor.cpp
2019-02-22 14:38:13 -08:00
91e05575a2
Rename OldTLogServer -> OldTLogServer_4_6
2019-02-19 22:18:10 -08:00
3f0fd2a20c
Use fwd decls in WorkerInterface
...
Also WorkerInterface.h -> WorkerInterface.actor.h
2019-02-19 15:16:59 -08:00
0bb60e5a3b
Use proper fwd decl in NativeAPI
...
Also NativeAPI.h -> NativeAPI.actor.h
2019-02-19 15:16:59 -08:00
065a45e05f
Merge branch 'master' into feature-fix-force-recovery
...
# Conflicts:
# fdbclient/ManagementAPI.actor.cpp
# fdbserver/ClusterController.actor.cpp
# fdbserver/workloads/KillRegion.actor.cpp
2019-02-18 17:09:06 -08:00
3247d59498
partially restored an optimization on remote storage servers where a behind storage server will keep less data in memory. This optimization was fully maintained on the primary storage servers, but remote storage servers can only use a version which is known to be durable on all remote transaction logs
2019-02-18 16:47:38 -08:00
e05b53d755
Merge remote-tracking branch 'apple/master' into task/tls-upgrade
2019-02-15 20:37:07 -08:00
7897616164
Fix wait failure bug on cluster controller
...
The setDistributor() sets an AsyncVar and then runs waitFailureClient. This
ordering is wrong because the AsyncVar::set triggers the other loop to run
first, which will wait on Never(). The correct code should wait on the Future
returned by the waitFailureClient.
2019-02-14 16:37:16 -08:00
Jingyu Zhou