wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
17,426 Commits 36 Branches 335 Tags 297 MiB
Commit Graph

1010 Commits

Author SHA1 Message Date
Sreenath Bodagala 246f035afe Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype 2021-10-18 19:07:33 +00:00
Yao Xiao e543570343
Add locality cache size to TransactionMetrics. (#5758) 
Co-authored-by: Yao Xiao <yaoxiao@Yaos-MacBook-Pro.local>
 2021-10-12 13:10:15 -07:00
Markus Pilman 5066d67dec
Merge pull request #5715 from sfc-gh-ljoswiak/fixes/tracing-various 
Add knob to control distributed trace recording percentage
 2021-10-11 14:03:53 -06:00
Markus Pilman 4fbc193e19 fix priority 2021-10-09 16:40:24 -06:00
Markus Pilman 9077e15b11 Address review comments 2021-10-09 16:40:24 -06:00
Markus Pilman 4e723ba3d9 Apply suggestions from code review 
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
 2021-10-09 16:40:24 -06:00
Markus Pilman ffdba4a133 assert we never get a stale read version 2021-10-09 16:40:24 -06:00
Dan Lambright f68d018a06 conflicts 2021-10-08 12:39:39 -04:00
Yao Xiao efd75ccc74
Add histograms to client GRV batcher. (#5727) 2021-10-07 13:30:25 -07:00
Lukas Joswiak e034f66fc2 Add knob to control trace recording percentage 2021-10-01 16:30:46 -07:00
A.J. Beamon 32a834b0c2 The computation of tag throttling backoff was taking a minimum with a knob for how long a throttle lasts. When this knob was buggified to 0 in simulation, the backoff would then be 0, and this could cause some transactions to get stuck in a retry loop that failed to advance time. Eventually the test would timeout or potentially run out of memory. 2021-09-23 10:03:57 -07:00
Vaidas Gasiunas 16171d8252 Refactoring well-known endpoint registration 
- List all well-known endpoints of FDB in a single enum
- Identify well-known endpoints by plain IDs
 2021-09-21 11:05:31 -06:00
Sreenath Bodagala d4f9a09a4f - Address test failures 2021-09-17 21:21:44 +00:00
Sreenath Bodagala 2aa3b44d4e Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype 
- Conflicts:
	fdbserver/LogSystem.h
	fdbserver/LogSystemConfig.h
	fdbserver/TagPartitionedLogSystem.actor.cpp

- Files modified during merge:

modified:   fdbserver/LogSystem.cpp
modified:   fdbserver/LogSystemConfig.cpp
 2021-09-17 19:36:18 +00:00
Xiaoge Su abf73047ca Enforce std:: specifier rather than using namespace 2021-09-16 19:40:28 -07:00
Sreenath Bodagala 35621a255e - Address review comments 2021-09-16 12:00:26 -04:00
Sreenath Bodagala ffbdc21ca8 - Address review comments 2021-09-16 12:00:26 -04:00
Sreenath Bodagala 852fc96200 Address simulation test failures caused by: 
- Assertion failures in MoveKeys.actor.cpp
- Wrong results returned by getRange()

Changes:

DatabaseContext.h, NativeAPI.actor.[h,cpp]:
- Introduce a new flag, TransactionInfo::readVersionObtainedFromGrvProxy.
- Set this flag to true by default, and clear it when the read version of a
transaction is explicitly set (by using setVersion()).
- Modify getLatestCommitVersions() to not populate "latestCommitVersions" if
this flag is not set. (This will cause storage server to read at the specified
read version.)

- Modify getRange() actor to always use the specified version as the read
version (except when the specified version is latestVersion).

- Modify waitForCommittedVersion(), getRawVersion(), and getConsistentReadVersion()
to update local version vector cache after receiving GetReadVersionReply.

IClientApi.h, IConfigTransaction.h, ISingleThreadTransaction.h,
MultiVersionTransaction[.actor].[h,cpp], ThreadSafeTransaction.[h,cpp],
ApiWorkload.h:
- Add methods to get the spanID of a transaction and also the version vector
cached in a transaction. (Likely to be useful for debugging simulation test
failures.)

VersionVector.h:
- Update "maxVersion" when populating/applying a delta. (Note that empty
mutation messages only update VersionVector::maxVersion.)

BackupWorker.actor.cpp:
- Update local version vector cache after receiving GetReadVersionReply message.

Status.actor.cpp:
- Update local version vector cache and
TransactionInfo::info.readVersionObtainedFromGrvProxy after setting the
read version.
 2021-09-16 12:00:26 -04:00
Josh Slocum 6673540553 TSS Metrics fix 2021-09-14 13:15:05 -05:00
A.J. Beamon afd8df4bb9 Merge commit '7f2b6c2c7ec2efcd7339e3c6531e4906322477c3' into network-thread-busyness-burstiness-score 2021-09-03 09:48:11 -07:00
A.J. Beamon 2fa97f60fe
Update fdbclient/NativeAPI.actor.cpp 
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
 2021-08-31 08:53:37 -07:00
Dan Lambright 8689e1f106 merge with master 2021-08-30 15:29:08 -04:00
FDB Formatster 2c788c233d apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-08-27 17:07:47 -07:00
A.J. Beamon 2d6fb93316 Compute a burstiness score based on the max busyness interval in a measurement period. 2021-08-27 16:29:15 -07:00
Chaoguang Lin b9628bdb50 Merge branch 'master' of github.com:apple/foundationdb into add-flow-db-interface 2021-08-17 20:24:41 +00:00
Chaoguang Lin 53e83cf653 Remove change to Database 2021-08-17 20:10:43 +00:00
Chaoguang Lin 179c313e08 Use DatabaseContext instead of Database in template functions 2021-08-17 20:02:25 +00:00
hao fu 925c4a935d Log size of locationCache in client 
We observed that multi-threaded client has more memory usage,
and we would like to understand it better.

One suspect is that this shard-to-SS cache is playing a role,
logging the size of it to get better vision.
 2021-08-16 10:31:14 -07:00
Josh Slocum e444d3781c Various TSS improvements from snowblower testing 2021-08-13 10:24:15 -05:00
Andrew Noyes 28f3ffd2ec Fix a crash in debug mode 
For unknown reasons, The previous implementation crashed while
(according to valgrind and ubsan) trying to interpret uninitialized memory as a
shared_ptr. This only happened when CMAKE_BUILD_TYPE=Debug. This change
works for debug mode and also still addresses this TSAN issue the
original change addressed.
 2021-08-12 16:19:20 -07:00
Lukas Joswiak a605fb3852
Merge pull request #5026 from sfc-gh-ljoswiak/fixes/alp6 
Actor sampling
 2021-08-11 13:44:17 -07:00
Sreenath Bodagala 2de3a1947e - Address an assertion failure in waitForVersion() 
Specify the correct "readVersion" when finding the latest commit versions
of storage server replicas (which will serve the read request).
 2021-08-11 15:36:35 +00:00
Chaoguang Lin 618c72f7e2 Draft to use template to rewrite ThrottleApi 2021-08-11 01:07:36 +00:00
Steve Atherton e5e8a56b66 Merge branch 'master' of https://github.com/apple/foundationdb into debugging-cleanup 
# Conflicts:
#	fdbbackup/FileDecoder.actor.cpp
 2021-08-09 23:36:57 -07:00
Andrew Noyes ecc078a42f Fix TSAN warnings 
Avoid doing thread-unsafe things in the ReadYourWritesTransaction
default constructor since we now will be calling it in a foreign thread.

Here's the warning we're fixing:

Write of size 8 at 0x7b6800000398 by thread T1:
    #0 N2::Net2::updateNow() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:192:33 (libfdb_c.so+0xe74568)
    #1 N2::Net2::run() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:1459:3 (libfdb_c.so+0xe74568)
    #2 runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:2036:13 (libfdb_c.so+0x575004)
    #3 ThreadSafeApi::runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:445:3 (libfdb_c.so+0xcda9cc)
    #4 MultiVersionApi::runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/MultiVersionTransaction.actor.cpp:1741:20 (libfdb_c.so+0x4eab8d)
    #5 fdb_run_network /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:130:2 (libfdb_c.so+0x4bb425)
    #6 decltype(std::__1::forward<int (*)()>(fp)()) std::__1::__invoke<int (*)()>(int (*&&)()) /usr/local/bin/../include/c++/v1/type_traits:3899:1 (fdb_c_unit_tests+0x389816)
    #7 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>&, std::__1::__tuple_indices<>) /usr/local/bin/../include/c++/v1/thread:280:5 (fdb_c_unit_tests+0x389816)
    #8 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()> >(void*) /usr/local/bin/../include/c++/v1/thread:291:5 (fdb_c_unit_tests+0x389816)

  Previous read of size 8 at 0x7b6800000398 by main thread:
    #0 N2::Net2::now() const /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:160:39 (libfdb_c.so+0xe7ea5d)
    #1 Span::Span(UID, Location, std::initializer_list<UID> const&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Tracing.h:38:41 (libfdb_c.so+0x5837b4)
    #2 Transaction::Transaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:4092:64 (libfdb_c.so+0x5837b4)
    #3 ReadYourWritesTransaction::ReadYourWritesTransaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ReadYourWrites.h:134:2 (libfdb_c.so+0x7cb4a8)
    #4 ISingleThreadTransaction::allocateOnForeignThread(ISingleThreadTransaction::Type) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ISingleThreadTransaction.cpp:29:17 (libfdb_c.so+0xcf785f)
    #5 ThreadSafeTransaction::ThreadSafeTransaction(DatabaseContext*, ISingleThreadTransaction::Type) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:149:23 (libfdb_c.so+0xcc9540)
    #6 ThreadSafeDatabase::createTransaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:50:37 (libfdb_c.so+0xcc4b1c)
    #7 fdb_database_create_transaction /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:332:2 (libfdb_c.so+0x4bce1d)
    #8 fdb::Transaction::Transaction(FDB_database*) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/fdb_api.cpp:111:24 (fdb_c_unit_tests+0x3925d0)
    #9 _DOCTEST_ANON_FUNC_10() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/unit_tests.cpp:250:19 (fdb_c_unit_tests+0x32a7a8)
    #10 doctest::Context::run() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/doctest/src/doctest/doctest/doctest.h:6112:21 (fdb_c_unit_tests+0x3262e4)
    #11 main /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/unit_tests.cpp:2286:20 (fdb_c_unit_tests+0x36303d)
 2021-08-09 11:16:52 -07:00
Steve Atherton 2bf7b9b7a9 Remove many calls to .toString(), .printable(), and printable(...) from TraceEvent .detail() values to avoid double-escaping characters such that the same values look different in different events and can't be easily searched for. Also fixed a few warnings found by clang. 2021-08-08 00:04:54 -07:00
Sreenath Bodagala a081c0baa5 Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype 2021-08-05 22:40:32 +00:00
Andrew Noyes 39eff8c569 Add trace_partial_file_suffix network option 2021-08-03 09:43:42 -07:00
Lukas Joswiak 5dc9a97230 Merge branch 'master' into fixes/alp6 2021-08-01 20:42:52 -07:00
sfc-gh-tclinkenbeard c74047c665 Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings 2021-07-28 11:51:02 -07:00
A.J. Beamon 9f571255bd
Merge pull request #5271 from sfc-gh-ajbeamon/do-not-reset-committed-transaction 
Do not partially reset a transaction when it is committed or fails to commit with an error
 2021-07-28 09:02:54 -07:00
Lukas Joswiak 3eed4084e2 Merge branch 'master' into fixes/alp6 2021-07-27 11:26:53 -07:00
Lukas Joswiak 59d535149e Merge branch 'master' into fixes/alp6 2021-07-27 10:07:18 -07:00
Lukas Joswiak e9a1679467 Disable sampling everywhere except fdbserver 2021-07-27 09:53:23 -07:00
Steve Atherton 507c1f11e3 Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use. 2021-07-26 19:55:10 -07:00
A.J. Beamon 34f82e7a15 Do not partially reset a transaction when it is committed or fails to commit with an error. 2021-07-26 09:51:44 -07:00
sfc-gh-tclinkenbeard 64dc1dc185 Fix -Wreorder-ctor warnings in NativeAPI.actor.cpp and several other files 2021-07-24 00:23:06 -07:00
Dan Lambright 1bb01ef591 use version vector in GetRange 2021-07-22 10:49:08 -04:00
sfc-gh-tclinkenbeard 6f81155784 Merge remote-tracking branch 'origin/master' into const-serverdbinfo 2021-07-20 10:18:40 -07:00
Steve Atherton f596a81073 Rename ::TRUE and ::FALSE in BooleanParams to ::True and ::False so as to not conflict with the TRUE and FALSE macros provided by the Windows and MacOS SDKs. 2021-07-17 00:11:40 -07:00