50e1e03130
fix: for configurations with anti-quorums to work, the push actors need to be put in the proxy’s actor collection
2018-06-18 15:25:54 -07:00
0913368651
added usable_regions to specify if we will replicate into a remote region
...
remote replication defaults to the primary replication
removed remote_logs, because they should be specified as an override in the regions object
2018-06-17 19:31:15 -07:00
f637c680f1
fix: populateSatelliteTagLocations was broken
...
fix: satellites do not index the upgraded locality
2018-06-17 13:29:17 -07:00
6931a00993
satellite log push locations are static per tag, which will reduce the number of tags each satellite log has to index, and reduce the proxy cpu when calculating push locations
2018-06-16 17:39:02 -07:00
f694f7c9ca
removed hasBestPolicy
2018-06-15 12:36:19 -07:00
0d87186821
use a specific locality for satellites
2018-06-15 11:06:38 -07:00
1796e00149
do not pop tags from logs that are not indexing that tag
2018-06-14 12:55:33 -07:00
889889323e
The master will tell the cluster controller if it is going to take a long time to recruit new logs in its DC; the cluster controller can determine if the other DC would be better and recruit there.
...
The cluster controller will not switch to the other data center if remote logs are too far behind.
We will not recruit in DCs with negative priority.
2018-06-13 18:14:14 -07:00
8dfda1e57b
fixed another trace event
2018-06-11 12:53:07 -07:00
372ed67497
Merge branch 'master' into feature-remote-logs
...
# Conflicts:
# fdbserver/DataDistribution.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/TagPartitionedLogSystem.actor.cpp
2018-06-11 11:34:10 -07:00
b60264024a
fix: we need to copy the txsTag on satellite logs
2018-06-10 20:30:44 -07:00
8a24bf6124
describe did not list all the log sets
2018-06-10 12:38:50 -07:00
e5488419cc
Attempt to normalize trace events:
...
* Detail names now all start with an uppercase character and contain no underscores. Ideally these should be head-first camel case, though that was harder to check.
* Type names have the same rules, except they allow one underscore (to support a usage pattern Context_Type). The first character after the underscore is also uppercase.
* Use seconds instead of milliseconds in details.
Added a check when events are logged in simulation that logs a message to stderr if the first two rules above aren't followed.
This probably doesn't address every instance of the above problems, but all of the events I was able to hit in simulation pass the check.
2018-06-08 11:11:08 -07:00
c519339adb
avoid peeking from logs that do not match the tag’s locality
2018-06-01 18:42:48 -07:00
81c7bddaf8
fix: must check for log router errors while waiting on satellite replies because the recruitmentID will not be updated if it threw an error
2018-05-06 18:15:12 -07:00
8371afb565
fix: log routers need to know if the log system is stopped to determine how they should peek the last log generation
2018-05-05 17:56:00 -07:00
e8ea02e054
fix: storage servers need to fail if they can no longer peek data
2018-05-05 17:19:59 -07:00
e1e43cff28
endEpoch implemented using getDurableVersion
2018-04-30 18:32:04 -07:00
5143871fed
passed debug ids into all versions of peek() to assist debugging
2018-04-30 13:36:35 -07:00
9cdabfed0e
added useful trace events
2018-04-29 18:54:47 -07:00
2e286b768d
fix: locality is needed for a logSet to call getPushLocations
...
fix: accidentally deleted allowPops assignment on the log router
2018-04-29 13:47:32 -07:00
dbdeeaa5cf
fix: log routers are given all the information they need to add remote tags in their initialization request
2018-04-28 18:04:57 -07:00
33fa8f2cac
fix: make sure log routers only add remote tags from the correct log set
2018-04-28 15:04:13 -07:00
23c0249d80
fix: old log routers tags must be available at the best location in the new generation
2018-04-28 11:13:10 -07:00
32e9ea3bb4
fix: recruited the wrong number of log routers
2018-04-26 22:22:15 -07:00
d72087bfd3
fix: we may not be able to recruit enough log routers, in this case put multiple log routers on the same worker, but also properly rank this configuration lower in better master exists
2018-04-26 22:18:07 -07:00
c7fd85243b
fix: passed the wrong argument value
2018-04-26 13:25:27 -07:00
0dd6931223
fix: remote recruitment must still wait for old log routers to be recruited since they are not needed by the newly recruited logs to finish recovery
2018-04-26 12:55:28 -07:00
721aaa2a6b
fix: we need to monitor old log routers for failures before recovery is complete
...
fix: after configuring out of fearless remote logs will not have all the data until the new configuration
2018-04-26 10:59:21 -07:00
a2b62e15ea
fix: only peek to peekEnd()
2018-04-25 19:56:50 -07:00
7e434348ce
fix: storage servers did not properly pull data when configuring from a fearless setup to a non-fearless setup
2018-04-25 18:20:28 -07:00
4119a1c5d5
do not add cursors for log sets that have no data
2018-04-24 22:06:10 -07:00
95855dbfc4
correctly filter locality data
2018-04-24 18:14:34 -07:00
35b2ca820a
fix: certain tlog errors during remote recovery could fail to kill the master, the master could have a reference counting cycle with its actor collection
2018-04-24 16:10:14 -07:00
3ec09ce9f6
fix: only peekSingle needs to throw worker_removed, because tlogs have other ways to get notified they are no longer needed
...
fix: we need to wait until tags are popped past recoveredAt instead of unrecovered before
2018-04-23 16:43:08 -07:00
126fc53d10
fix: the start version for peek cursors that merge with multiple log sets is the maximum of the individual start versions
2018-04-23 12:42:51 -07:00
73597f190e
fix: new tlogs are initialized with exactly the tags which existed at the recovery version
2018-04-22 20:28:01 -07:00
ef23136809
fix: ensure the logSystemConfig is updated with newly recruited log routers
2018-04-22 11:54:39 -07:00
fceec020de
fix: use the known committed version if the last generation primary logs were in the same data center as this generation
...
the known committed version in end epoch is the maximum seen in all responses regardless of log set
2018-04-22 11:14:13 -07:00
c3a344d44e
fix: do not choose a remote start version past the start of the locked logs
2018-04-21 16:03:28 -07:00
28a1fa9dc2
fix: we need to notify the old log system that its recruitmentID has changed
2018-04-21 12:57:00 -07:00
a6d9e889f0
a cleaner solution to preventing tlogs from peeking log routers
2018-04-20 13:25:22 -07:00
f5c3417905
fix: prevent tlogs from peeking the wrong log routers
2018-04-20 00:30:37 -07:00
447c7bd15b
fix: log routers use durable known committed version at the time of the pop to determine what is safe to pop from their logs
...
fix: storage server does not advance its version across large version increase until it has data associated with the version
2018-04-18 12:07:29 -07:00
cef6c9b418
fix: the startVersion cannot be larger than the known committed version
2018-04-16 16:21:27 -07:00
e53f17a83a
fix: the newest log router needs to start where the last old one ends
2018-04-15 14:54:22 -07:00
5533016f1e
fix: tlogs are now initialized immediately, instead of when starting the core, this must be done to pop the log routers during recovery
...
fix: log router start version must be the same as remote log start version
2018-04-15 14:33:07 -07:00
041f5787fb
fix: peekLocal does not stop when a locality does not exist
...
fix: lock logs only stops on special or upgraded locality
fix: recruiting old log routers respects the passed in startVersion
2018-04-14 19:06:24 -07:00
c589630e53
fix: log router start version is based on the start version of the local logs
2018-04-12 18:14:23 -07:00
1af5ac0d9d
fix: a number of different problems prevented tlogs from using log routers during recovery
2018-04-12 15:20:54 -07:00
Evan Tschannen