wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,033 Commits 36 Branches 335 Tags 297 MiB
Commit Graph

228 Commits

Author SHA1 Message Date
Evan Tschannen 3f9d9d8b84 Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	cmake/FlowCommands.cmake
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/StorageServerInterface.h
#	fdbserver/DataDistributionTracker.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/Knobs.h
#	flow/Platform.cpp
#	versions.target
 2020-01-16 18:37:47 -08:00
Evan Tschannen da1be272cb fix: servers which opened the database would use the full list of proxies 2020-01-10 12:20:30 -08:00
Evan Tschannen ebcb2f79ed Merge branch 'master' of github.com:apple/foundationdb 2019-11-22 15:34:49 -08:00
Evan Tschannen 8d3ef89540 Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/MutationList.h
#	fdbserver/MasterProxyServer.actor.cpp
#	versions.target
 2019-11-14 15:49:56 -08:00
negoyal a4a0bf18f9 Merging with Master. 2019-11-12 13:01:29 -08:00
Evan Tschannen 1e5677b55a increase the priority of reboot and recruitment requests 2019-11-11 15:17:11 -08:00
Jon Fu 103cc37a35 added datahall kill and option to target a specific datahall/dc/machine id 2019-10-23 14:19:17 -07:00
Alex Miller 1eb3a70b96 Spill SharedTLog when there's more than one. 
When switching between spill_type or log_version, a new instance of a
SharedTLog is created in the transaction log processes.  If this is done
in a saturated database, then doubling the amount of memory to hold
mutations in memory can cause TLogs to be uncomfortably close to the 8GB
OOM limit.

Instead, we now thread which UID of a SharedTLog is active, and the
other TLog spill out the majority of their mutations.

This is a backport of #2213 (fef89aa1) to release-6.2
 2019-10-17 01:24:50 -07:00
Alex Miller 77c72de176 Comment variable and code style fix 
Co-Authored-By: Jingyu Zhou <jingyuzhou@gmail.com>
 2019-10-07 18:08:27 -07:00
Alex Miller a34a009bf0 Shuffle member initialization in constructor. 2019-10-07 18:08:27 -07:00
Alex Miller 71af24dff3 Fix a bug that would cause active logs to spill aggressively 
And add some useful logging about when things do or do not spill.
 2019-10-07 18:08:27 -07:00
Alex Miller 1d8a7e5af7 Spill SharedTLog when there's more than one. 
When switching between spill_type or log_version, a new instance of a
SharedTLog is created in the transaction log processes.  If this is done
in a saturated database, then doubling the amount of memory to hold
mutations in memory can cause TLogs to be uncomfortably close to the 8GB
OOM limit.

Instead, we now thread which UID of a SharedTLog is active, and the
other TLog spill out the majority of their mutations.
 2019-10-07 18:08:27 -07:00
Alex Miller 9f9d2dff42 Point both spill types at the same SharedTLog for >=V5 
This actually results in TLog generations of different spill types in
the same disk queue.  We also drop the LS_ parameter from the filename
to signify that there is no log spill configuration per file anymore.
 2019-10-03 01:45:50 -07:00
Alex Miller 6742222084 Make TLogServer able to spill by value and by reference 
...and test it in simulation, but not combined yet.

It turns out that because of txsTag, we basically had to support
spill-by-value anyway.  Thus, if we treat all tags like txsTag when
spilling and peeking, then we have an easy way to bring the two spilling
types back into one implementation.
 2019-10-03 01:45:10 -07:00
Alex Miller 24c46337e1 Advance TLogVersions for 7.0 while adding a V5 that is TLogServer 
Advancing the MIN_RECRUITABLE and DEFAULT is just following the standard
progression for 7.0.  It was convenient to do while adding the V5 so
that we can hook TLogServer back into being used.
 2019-10-03 01:43:01 -07:00
Alex Miller 60fb04ca68 Fork TLogServer into TLogServer_6_2 
This prepares us for incoming modifications to the TLog that can't
easily coexist with our current on-disk state.
 2019-10-03 01:41:25 -07:00
Evan Tschannen b495cc697b Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	versions.target
 2019-09-13 09:25:08 -07:00
Evan Tschannen 945cff1e5b the cluster controller caches the serialization of serverDBInfo, to avoid regenerating it many times 2019-09-10 14:27:22 -07:00
A.J. Beamon 41fd3d9467 Merge branch 'master' into remove-unused-ssi-get-version 
# Conflicts:
#	fdbclient/StorageServerInterface.h
#	fdbserver/storageserver.actor.cpp
 2019-09-10 08:11:43 -07:00
Evan Tschannen 24aad14f06 Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	versions.target
 2019-08-30 17:23:58 -07:00
sramamoorthy b3277f2982 Fix #2009 posix compliant args for snapshot binary 2019-08-30 12:54:09 -07:00
Jingyu Zhou 3d0bfbd66e Revert a change moving promise out as state variable 
The "recovery" is constructed in a loop, so moving out is wrong.
 2019-08-27 11:34:43 -07:00
A.J. Beamon 64ce0c3285 Remove the unused getVersion from StorageServerInterface. 2019-08-26 13:53:54 -07:00
Jingyu Zhou 17e2630b21 Move state variable to the head of the function 2019-08-22 14:21:13 -07:00
Jingyu Zhou 26444cad95 Fix another vector destructor caused broken promise 2019-08-22 14:12:05 -07:00
Jingyu Zhou 085706c70c Fix a clang vector destruction issue. 
This addresses issue #1862.
 2019-08-22 10:21:13 -07:00
A.J. Beamon 14648e20f9
Merge pull request #1901 from ajbeamon/data-distribution-receives-bytes-input-rate 
Send bytes input rate to data distribution
 2019-07-30 15:01:36 -07:00
sramamoorthy 9afd162e2f remove snap v1 related code 2019-07-25 17:29:31 -07:00
A.J. Beamon b91795d288 Send bytes input rate to DD. 2019-07-25 16:27:32 -07:00
sramamoorthy a65c9f92ed get rid of all timeouts and other changes 2019-07-24 15:36:28 -07:00
sramamoorthy 31c010b393 few minor fixes 2019-07-24 15:36:28 -07:00
sramamoorthy 080b3da322 includeCancelled for workerSnapCreate 2019-07-24 15:36:28 -07:00
sramamoorthy 8f1f0c0435 snap v2: worker and other helper related changes 2019-07-24 15:36:28 -07:00
Evan Tschannen 7ad0d1a12b Merge branch 'master' into feature-proxy-forward 
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
 2019-07-09 17:26:15 -07:00
Meng Xu cce00bb413
Merge pull request #1808 from ajbeamon/improved-transaction-metrics 
Improve TransactionMetrics
 2019-07-09 16:46:17 -07:00
Evan Tschannen d032d7fcf9 fix: if we get a broken_promise from the actor, wait to get the real error from the store 2019-07-09 16:37:54 -07:00
Vishesh Yadav eabc610daa
Merge pull request #1813 from alexmiller-apple/log-version-4 
Add a TLogVersion::V4
 2019-07-09 08:42:20 -07:00
Alex Miller d2ef84a8f9 Add a TLogVersion::V4 
And refactor some code to make adding more TLogVersions easier.
 2019-07-08 22:22:45 -07:00
A.J. Beamon a5a6f8431c Add a random UID to TransactionMetrics in case a client opens multiple connections and also a field to indicate whether the connection is internal. Convert some of the metrics to our Counter object instead of running totals. 2019-07-08 14:01:04 -07:00
Alex Miller ea6898144d Merge remote-tracking branch 'upstream/master' into flowlock-api 2019-07-03 20:44:15 -07:00
A.J. Beamon 8c10d832a1 Add coordinator role in trace events 2019-07-03 11:09:36 -07:00
mengranwo 0b9cd18fb4 checking cluster is healthy or not during recovery process(for storage engine), if healthy, delete data files and join as new 2019-07-01 16:09:51 -07:00
Alex Miller bf883d7055 Merge remote-tracking branch 'upstream/master' into flowlock-api 2019-06-25 14:26:50 -07:00
Alex Miller 7a500cd37f A giant translation of TaskFooPriority -> TaskPriority::Foo 
This is so that APIs that take priorities don't take ints, which are
common and easy to accidentally pass the wrong thing.
 2019-06-25 02:47:35 -07:00
Evan Tschannen 37c1df2491
Merge pull request #1705 from bnamasivayam/suspend-process 
Extend RebootRequest API to include time to suspend the process befor…
 2019-06-20 17:36:25 -07:00
Balachandar Namasivayam 5eb833759e Extend RebootRequest API to include time to suspend the process before reboot. This is intended to be used for testing purposes to simulate failures. 2019-06-14 11:35:38 -07:00
Parallels 773f52d0a1 Merge remote-tracking branch 'upstream/master' into cloexec 2019-06-03 15:43:32 -07:00
sramamoorthy d3a179b6f9 Multiple bug fixes 
- wait for snapTLogFailKeys in a loop, otherwise in some race
  condition it can cause a false assert
- in single region, there does not seem to be a guarantee of
  tagLocalityListKey for a given DC ID, avoiding that assert for now
- to find the workers that are coordinators, looking up by primary
  address is not sufficient in some cases, hence looking by both
  primary and secondary address
- test make files to reflect the location of the new test cases
 2019-05-28 22:07:46 -07:00
sramamoorthy 936ffc2dde rebase related changes 2019-05-28 22:07:46 -07:00
sramamoorthy ec7834e2f7 code re-orgnaization and address comments 2019-05-28 22:07:46 -07:00