f045c041eb
fix: if a storage server already exists in a remote region after converting to fearless, it did not receive mutations between the known committed version and the recovery version
2018-11-02 14:11:39 -07:00
1b5d28386a
fix: the Tlog would not update the durable version properly when version_sizes was empty
2018-11-02 13:05:54 -07:00
ecddeab2ae
fixed review comments; demote killRegionCycle test for now
2018-10-08 10:39:39 -07:00
15ce215c1b
fix: parallel peek requests leaked memory
2018-10-02 17:28:39 -07:00
a24eadd73a
fix: for remote logs, their known committed version cannot be set to 1, because they can be used when their durable version is 0, leading to a known committed version being greater than a queue committed version
2018-09-28 12:17:21 -07:00
77e2fb787e
Merge branch 'release-6.0' into feature-fix-forced-recovery
2018-09-21 14:55:37 -07:00
31d0b0315f
fix: tlog spill policy would spill everything when it wanted to spill nothing
...
use a flow lock to protect updatePersistData and initPersistentState from committing simultaneously
2018-09-20 15:33:38 -07:00
270b1b24a6
fix: we have to use durableKnownCommittedVersion, because the is the true lower bound on the recovery version of the remote logs
...
fixed a compiler error
2018-09-18 16:29:03 -07:00
200e65fe61
added a workload which tests killing an entire region, and recovering from the failure with data loss.
...
fix: we cannot pop the txs tag from remote logs until they have a full copy of the txnStateStore
fix: we have to modify all of history, we cannot stop after finding a local remote
2018-09-17 18:32:39 -07:00
df406a340e
Merge pull request #742 from ajbeamon/roles-in-trace-events
...
Add the roles running on a process as a field on trace events in the …
2018-09-05 16:08:12 -07:00
2de0b5d6d7
Add the roles running on a process as a field on trace events in the form of a comma delimited string of role abbreviations.
2018-09-05 15:06:14 -07:00
1022e0a5c6
added yields to the log router and tlogs after processing a version
2018-09-04 17:16:44 -07:00
84e1f7b2b5
added overhead bytes durable to complement overhead bytes input
2018-08-21 22:35:04 -07:00
74f7412975
added separate logging for overhead bytes
2018-08-21 22:18:38 -07:00
d7c01f0419
added a separate knob for tlog’s recoverMemoryLimit
2018-08-21 21:11:23 -07:00
7c5d414f7b
fix: during destruction logData could attempt to dereference tLogData after it has been deleted
2018-08-09 12:38:35 -07:00
c757c68bfa
fix: nextVersion needs to be set to logData->version if version_sizes is empty
2018-08-04 23:53:37 -07:00
fec285146c
significant cpu optimization in update storage
2018-08-04 12:36:48 -07:00
be1a4d74c7
tlogs serve reads to log routers at a low priority, to prevent them from using all their resources catching up a remote dc that has been down for a long time
...
increase the amount of memory ratekeeper budgets for tlogs so that there is a gap after the spill threshold to prevent temporarily overshooting the budget
2018-08-04 10:31:30 -07:00
71f89f372f
changed a trace event name to avoid scope type mismatch on the tag field
2018-08-03 15:53:38 -07:00
2619234477
Merge branch 'release-5.2' into release-6.0
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
2018-08-03 11:40:24 -07:00
501033c5af
fix: tlog spilling on a stopped log was only making one version durable at a time
2018-08-03 11:38:12 -07:00
1c29275672
call all methods which could disable a trace event before it is initialized. In practice this means calling .error first, then .suppressFor, then all your details.
2018-08-01 14:30:57 -07:00
0f59dc4086
fix: do not write to the persistent queue when we are terminated, which could happen if shutdown was caused by setting a promise in the asyncPullData loop
2018-07-13 17:01:31 -07:00
cd63c7a7cc
added a buffered cursor, which efficiently merges lots of peek cursors
2018-07-12 12:09:48 -07:00
6b40f2764d
fix: off by one error on popping missing tags
2018-07-09 15:43:22 -07:00
2718176927
fix: remote logs did not pop all of the data for removed logs on recovery because data for the missing tag was not recorded yet at the time of recovery
2018-07-06 16:10:41 -07:00
507b3bacb0
fix: kill all tlogs in one region prevents the remote logs from recovering in that region, do not allow that to prevent us from configuring usable_regions=1.
...
added more recovery states.
2018-07-05 00:08:51 -07:00
2987f85177
fix: known committed version must be updated before creating the tlogQueueEntryRef
2018-06-26 23:21:30 -07:00
6e19622872
fix: durableKnownCommittedVersion was incorrect because we could update knownCommittedVersion before pushing a TLogQueueEntry with that known committed version
2018-06-26 18:02:55 -07:00
2ec8744ab3
fix: parallel get more needs to verify the begin version matches the end of the previous request, because when a peek cursor expires we lose all history, so the same sequence number could start at different versions
2018-06-25 11:15:49 -07:00
68ac3bdc4c
log routers now calculate a precise version to pop for their log router tag
2018-06-21 15:29:46 -07:00
f755961c42
use parallelGetMore during log recovery
2018-06-20 17:04:06 -07:00
df4c445e25
fix: we need to return from commit when stopped
2018-06-18 22:12:46 -07:00
403fb5a2e9
removed unnecessary suppressFor
2018-06-18 17:59:29 -07:00
b79feaddd3
added a hard memory limit to the TLog to prevent it from running out of memory. Because remote logs are not ratekeeper controlled this is their only protection
2018-06-18 17:22:40 -07:00
7aef5ec6f1
got rid of persistUnrecoveredBefore
...
added persistLocality
2018-06-17 14:44:33 -07:00
f637c680f1
fix: populateSatelliteTagLocations was broken
...
fix: satellites do not index the upgraded locality
2018-06-17 13:29:17 -07:00
0d87186821
use a specific locality for satellites
2018-06-15 11:06:38 -07:00
feb8578c06
fix: only flush and exit in simulation
2018-06-14 13:48:30 -07:00
fcfa00928b
Make RecoveryState an enum class.
...
This means that all the == 7 or != 0 checks go away, and explicit names must be used.
2018-06-12 16:50:25 -07:00
372ed67497
Merge branch 'master' into feature-remote-logs
...
# Conflicts:
# fdbserver/DataDistribution.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/TagPartitionedLogSystem.actor.cpp
2018-06-11 11:34:10 -07:00
588eaf4b36
fix: previous delay 0 could still cause us to recruit a tlog before processing disk errors
2018-06-11 11:26:30 -07:00
a5c2a8ee8a
fix: allow disk errors to cancel the actor before recruiting logs
2018-06-10 20:27:19 -07:00
e5488419cc
Attempt to normalize trace events:
...
* Detail names now all start with an uppercase character and contain no underscores. Ideally these should be head-first camel case, though that was harder to check.
* Type names have the same rules, except they allow one underscore (to support a usage pattern Context_Type). The first character after the underscore is also uppercase.
* Use seconds instead of milliseconds in details.
Added a check when events are logged in simulation that logs a message to stderr if the first two rules above aren't followed.
This probably doesn't address every instance of the above problems, but all of the events I was able to hit in simulation pass the check.
2018-06-08 11:11:08 -07:00
bf65e745a9
tlogs do not index tags for other localities
2018-06-01 22:51:08 -07:00
c519339adb
avoid peeking from logs that do not match the tag’s locality
2018-06-01 18:42:48 -07:00
1f0b519a73
Rename several variables in TLogServer.actor.cpp to follow our normal camel case conventions. I didn't rename every variable here, because some appear to be data structures (like a map) following the pattern keydesc_valuedesc, and I wasn't sure that the straightforward keydescValuedesc rename made sense. I did rename a couple of instances of these where it seemed reasonable, though.
2018-06-01 10:18:07 -07:00
d9c702a9e3
Merge release-5.1 into release-5.2
2018-05-30 09:09:55 -07:00
529bd34cf9
fix: when a tlog is stopped by another recruitment it no longer has the opportunity for commtingQueue to be set
2018-05-06 20:37:44 -07:00
Evan Tschannen