efc4cec53f
fixed a variety of bugs with change feeds
2021-10-09 19:24:01 -07:00
4fbc193e19
fix priority
2021-10-09 16:40:24 -06:00
9077e15b11
Address review comments
2021-10-09 16:40:24 -06:00
4e723ba3d9
Apply suggestions from code review
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2021-10-09 16:40:24 -06:00
ffdba4a133
assert we never get a stale read version
2021-10-09 16:40:24 -06:00
2b2d059039
Specify option to specifically skip grv cache for the background updater when using debug knob
2021-10-08 17:32:44 -04:00
b770936cde
move background updater actor initialization
2021-10-08 16:38:21 -04:00
ecdafba336
add debug traces to background grv updater
2021-10-08 15:17:28 -04:00
f68d018a06
conflicts
2021-10-08 12:39:39 -04:00
efd75ccc74
Add histograms to client GRV batcher. ( #5727 )
2021-10-07 13:30:25 -07:00
31664ddb78
Fixing change feed error retry in pop
2021-10-07 10:07:18 -05:00
54d5e8366d
fixing another change feed hang error
2021-10-06 16:04:01 -05:00
cc38c48123
Add debug knob to use grv cache for performance testing
2021-10-06 14:35:42 -04:00
bbe7c7ca9f
found change feed liveness bug
2021-10-06 12:32:16 -05:00
9a18cc8f41
Change debug time bound update logic to be located in masterserver
2021-10-05 17:31:31 -04:00
44a854772f
Merge branch 'master' of github.com:apple/foundationdb into jfu-grv-cache
2021-10-05 12:55:02 -04:00
ae31f01dc4
remove cache update from extractreadversion and set lower time bound to 0
2021-10-05 12:54:22 -04:00
d560eb1fea
debug time bounds using sim_validation
2021-10-04 14:12:31 -04:00
e034f66fc2
Add knob to control trace recording percentage
2021-10-01 16:30:46 -07:00
fa1fe5f08b
added blob worker rollbacks that handle (most) cases
2021-09-24 17:52:36 -05:00
a986679b7f
More bug fixes
2021-09-24 10:02:02 -05:00
32a834b0c2
The computation of tag throttling backoff was taking a minimum with a knob for how long a throttle lasts. When this knob was buggified to 0 in simulation, the backoff would then be 0, and this could cause some transactions to get stuck in a retry loop that failed to advance time. Eventually the test would timeout or potentially run out of memory.
2021-09-23 10:03:57 -07:00
4cd5d138ff
Fixed segfault involving missing futures and infinite looping in background actor
2021-09-22 13:13:35 -04:00
16171d8252
Refactoring well-known endpoint registration
...
- List all well-known endpoints of FDB in a single enum
- Identify well-known endpoints by plain IDs
2021-09-21 11:05:31 -06:00
d4f9a09a4f
- Address test failures
2021-09-17 21:21:44 +00:00
2aa3b44d4e
Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
...
- Conflicts:
fdbserver/LogSystem.h
fdbserver/LogSystemConfig.h
fdbserver/TagPartitionedLogSystem.actor.cpp
- Files modified during merge:
modified: fdbserver/LogSystem.cpp
modified: fdbserver/LogSystemConfig.cpp
2021-09-17 19:36:18 +00:00
abf73047ca
Enforce std:: specifier rather than using namespace
2021-09-16 19:40:28 -07:00
35621a255e
- Address review comments
2021-09-16 12:00:26 -04:00
ffbdc21ca8
- Address review comments
2021-09-16 12:00:26 -04:00
852fc96200
Address simulation test failures caused by:
...
- Assertion failures in MoveKeys.actor.cpp
- Wrong results returned by getRange()
Changes:
DatabaseContext.h, NativeAPI.actor.[h,cpp]:
- Introduce a new flag, TransactionInfo::readVersionObtainedFromGrvProxy.
- Set this flag to true by default, and clear it when the read version of a
transaction is explicitly set (by using setVersion()).
- Modify getLatestCommitVersions() to not populate "latestCommitVersions" if
this flag is not set. (This will cause storage server to read at the specified
read version.)
- Modify getRange() actor to always use the specified version as the read
version (except when the specified version is latestVersion).
- Modify waitForCommittedVersion(), getRawVersion(), and getConsistentReadVersion()
to update local version vector cache after receiving GetReadVersionReply.
IClientApi.h, IConfigTransaction.h, ISingleThreadTransaction.h,
MultiVersionTransaction[.actor].[h,cpp], ThreadSafeTransaction.[h,cpp],
ApiWorkload.h:
- Add methods to get the spanID of a transaction and also the version vector
cached in a transaction. (Likely to be useful for debugging simulation test
failures.)
VersionVector.h:
- Update "maxVersion" when populating/applying a delta. (Note that empty
mutation messages only update VersionVector::maxVersion.)
BackupWorker.actor.cpp:
- Update local version vector cache after receiving GetReadVersionReply message.
Status.actor.cpp:
- Update local version vector cache and
TransactionInfo::info.readVersionObtainedFromGrvProxy after setting the
read version.
2021-09-16 12:00:26 -04:00
6673540553
TSS Metrics fix
2021-09-14 13:15:05 -05:00
87fac6e301
adjust to changed function signature after merge
2021-09-13 13:04:55 -04:00
d971b63bbf
Merge branch 'master' of github.com:apple/foundationdb into jfu-grv-cache
2021-09-13 12:47:42 -04:00
cfbe5eca9c
Use notifieddouble to track GRV request time and include grvdelay in timing calculations
2021-09-13 11:57:19 -04:00
cefb66d64c
fixing a couple bugs
2021-09-10 12:52:33 -05:00
c2d1d1704f
Merge branch 'feature-range-feed' into blob_full
2021-09-10 11:21:52 -05:00
ac5b580e2d
Merge branch 'master' into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/StorageServerInterface.cpp
# fdbclient/StorageServerInterface.h
# fdbserver/ApplyMetadataMutation.cpp
# fdbserver/TLogServer.actor.cpp
# flow/error_definitions.h
2021-09-09 23:13:22 -07:00
1ce2457f7e
initial commit
2021-09-09 14:01:42 -04:00
eb76343dfb
Added blob granule reassignment and splitting
2021-09-08 14:09:14 -05:00
d9782bc29d
fixed a few bugs with merging change feeds on the client
2021-09-07 21:35:00 -07:00
f53ddf6720
fix: handle the case where I change feed is added while a fetch is in progress
2021-09-06 13:51:53 -07:00
af0792b1f5
fixes that allow change feeds to pass correctness in tests with no failures
2021-09-03 17:24:33 -07:00
b02e8d99c6
added a workload to test change feeds
2021-09-03 14:19:27 -07:00
8309c2ce26
filter mutations in change feeds to the requested range
2021-09-03 12:06:13 -07:00
afd8df4bb9
Merge commit '7f2b6c2c7ec2efcd7339e3c6531e4906322477c3' into network-thread-busyness-burstiness-score
2021-09-03 09:48:11 -07:00
4bbae59bb0
Fixed a few bugs and added a backup popping mechanism
2021-09-02 21:11:44 -07:00
27151b0831
support reads from range feeds that span multiple storage servers
2021-09-01 14:35:51 -07:00
46adada5ff
Cleaned up debugging and fixed a couple bugs
2021-08-31 12:30:43 -05:00
2fa97f60fe
Update fdbclient/NativeAPI.actor.cpp
...
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2021-08-31 08:53:37 -07:00
8689e1f106
merge with master
2021-08-30 15:29:08 -04:00
2c788c233d
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-08-27 17:07:47 -07:00
2d6fb93316
Compute a burstiness score based on the max busyness interval in a measurement period.
2021-08-27 16:29:15 -07:00
8d49c98a41
Added simulation workload for blob granules and fixed some bugs
2021-08-26 13:48:05 -05:00
714aca4f3c
Refactored blob granule reads to actually be in NativeAPI
2021-08-25 15:01:38 -05:00
b9628bdb50
Merge branch 'master' of github.com:apple/foundationdb into add-flow-db-interface
2021-08-17 20:24:41 +00:00
53e83cf653
Remove change to Database
2021-08-17 20:10:43 +00:00
179c313e08
Use DatabaseContext instead of Database in template functions
2021-08-17 20:02:25 +00:00
925c4a935d
Log size of locationCache in client
...
We observed that multi-threaded client has more memory usage,
and we would like to understand it better.
One suspect is that this shard-to-SS cache is playing a role,
logging the size of it to get better vision.
2021-08-16 10:31:14 -07:00
a278d2977a
renamed range feeds to change feeds, data distribution support almost complete
2021-08-13 14:27:15 -07:00
e444d3781c
Various TSS improvements from snowblower testing
2021-08-13 10:24:15 -05:00
28f3ffd2ec
Fix a crash in debug mode
...
For unknown reasons, The previous implementation crashed while
(according to valgrind and ubsan) trying to interpret uninitialized memory as a
shared_ptr. This only happened when CMAKE_BUILD_TYPE=Debug. This change
works for debug mode and also still addresses this TSAN issue the
original change addressed.
2021-08-12 16:19:20 -07:00
a605fb3852
Merge pull request #5026 from sfc-gh-ljoswiak/fixes/alp6
...
Actor sampling
2021-08-11 13:44:17 -07:00
2de3a1947e
- Address an assertion failure in waitForVersion()
...
Specify the correct "readVersion" when finding the latest commit versions
of storage server replicas (which will serve the read request).
2021-08-11 15:36:35 +00:00
94239098a5
added additional rangefeed commands to fdbcli
2021-08-10 23:59:05 -07:00
7891477a7c
added support for streaming range feeds
2021-08-10 21:28:25 -07:00
618c72f7e2
Draft to use template to rewrite ThrottleApi
2021-08-11 01:07:36 +00:00
e5e8a56b66
Merge branch 'master' of https://github.com/apple/foundationdb into debugging-cleanup
...
# Conflicts:
# fdbbackup/FileDecoder.actor.cpp
2021-08-09 23:36:57 -07:00
ed28aecde0
Merge branch 'master' into feature-range-feed
2021-08-09 20:40:55 -07:00
42ae870c84
added support for querying specific range feed versions
2021-08-09 20:39:28 -07:00
ecc078a42f
Fix TSAN warnings
...
Avoid doing thread-unsafe things in the ReadYourWritesTransaction
default constructor since we now will be calling it in a foreign thread.
Here's the warning we're fixing:
Write of size 8 at 0x7b6800000398 by thread T1:
#0 N2::Net2::updateNow() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:192:33 (libfdb_c.so+0xe74568)
#1 N2::Net2::run() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:1459:3 (libfdb_c.so+0xe74568)
#2 runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:2036:13 (libfdb_c.so+0x575004)
#3 ThreadSafeApi::runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:445:3 (libfdb_c.so+0xcda9cc)
#4 MultiVersionApi::runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/MultiVersionTransaction.actor.cpp:1741:20 (libfdb_c.so+0x4eab8d)
#5 fdb_run_network /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:130:2 (libfdb_c.so+0x4bb425)
#6 decltype(std::__1::forward<int (*)()>(fp)()) std::__1::__invoke<int (*)()>(int (*&&)()) /usr/local/bin/../include/c++/v1/type_traits:3899:1 (fdb_c_unit_tests+0x389816)
#7 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>&, std::__1::__tuple_indices<>) /usr/local/bin/../include/c++/v1/thread:280:5 (fdb_c_unit_tests+0x389816)
#8 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()> >(void*) /usr/local/bin/../include/c++/v1/thread:291:5 (fdb_c_unit_tests+0x389816)
Previous read of size 8 at 0x7b6800000398 by main thread:
#0 N2::Net2::now() const /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:160:39 (libfdb_c.so+0xe7ea5d)
#1 Span::Span(UID, Location, std::initializer_list<UID> const&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Tracing.h:38:41 (libfdb_c.so+0x5837b4)
#2 Transaction::Transaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:4092:64 (libfdb_c.so+0x5837b4)
#3 ReadYourWritesTransaction::ReadYourWritesTransaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ReadYourWrites.h:134:2 (libfdb_c.so+0x7cb4a8)
#4 ISingleThreadTransaction::allocateOnForeignThread(ISingleThreadTransaction::Type) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ISingleThreadTransaction.cpp:29:17 (libfdb_c.so+0xcf785f)
#5 ThreadSafeTransaction::ThreadSafeTransaction(DatabaseContext*, ISingleThreadTransaction::Type) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:149:23 (libfdb_c.so+0xcc9540)
#6 ThreadSafeDatabase::createTransaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:50:37 (libfdb_c.so+0xcc4b1c)
#7 fdb_database_create_transaction /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:332:2 (libfdb_c.so+0x4bce1d)
#8 fdb::Transaction::Transaction(FDB_database*) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/fdb_api.cpp:111:24 (fdb_c_unit_tests+0x3925d0)
#9 _DOCTEST_ANON_FUNC_10() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/unit_tests.cpp:250:19 (fdb_c_unit_tests+0x32a7a8)
#10 doctest::Context::run() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/doctest/src/doctest/doctest/doctest.h:6112:21 (fdb_c_unit_tests+0x3262e4)
#11 main /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/unit_tests.cpp:2286:20 (fdb_c_unit_tests+0x36303d)
2021-08-09 11:16:52 -07:00
bc9a0e1315
first attempt to add data distribution support for range feeds
2021-08-09 10:05:56 -07:00
2bf7b9b7a9
Remove many calls to .toString(), .printable(), and printable(...) from TraceEvent .detail() values to avoid double-escaping characters such that the same values look different in different events and can't be easily searched for. Also fixed a few warnings found by clang.
2021-08-08 00:04:54 -07:00
a081c0baa5
Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
2021-08-05 22:40:32 +00:00
39eff8c569
Add trace_partial_file_suffix network option
2021-08-03 09:43:42 -07:00
5dc9a97230
Merge branch 'master' into fixes/alp6
2021-08-01 20:42:52 -07:00
0989c28a6b
made range feeds durable on the storage server
2021-07-30 15:23:42 -07:00
e9409b02fe
fixed the build
2021-07-28 14:02:12 -07:00
c74047c665
Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings
2021-07-28 11:51:02 -07:00
9f571255bd
Merge pull request #5271 from sfc-gh-ajbeamon/do-not-reset-committed-transaction
...
Do not partially reset a transaction when it is committed or fails to commit with an error
2021-07-28 09:02:54 -07:00
406562b282
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
2021-07-27 17:04:58 -07:00
3eed4084e2
Merge branch 'master' into fixes/alp6
2021-07-27 11:26:53 -07:00
59d535149e
Merge branch 'master' into fixes/alp6
2021-07-27 10:07:18 -07:00
e9a1679467
Disable sampling everywhere except fdbserver
2021-07-27 09:53:23 -07:00
507c1f11e3
Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use.
2021-07-26 19:55:10 -07:00
34f82e7a15
Do not partially reset a transaction when it is committed or fails to commit with an error.
2021-07-26 09:51:44 -07:00
64dc1dc185
Fix -Wreorder-ctor warnings in NativeAPI.actor.cpp and several other files
2021-07-24 00:23:06 -07:00
1bb01ef591
use version vector in GetRange
2021-07-22 10:49:08 -04:00
6f81155784
Merge remote-tracking branch 'origin/master' into const-serverdbinfo
2021-07-20 10:18:40 -07:00
f596a81073
Rename ::TRUE and ::FALSE in BooleanParams to ::True and ::False so as to not conflict with the TRUE and FALSE macros provided by the Windows and MacOS SDKs.
2021-07-17 00:11:40 -07:00
81001edb2e
- Propagate version vector deltas between processes
2021-07-15 19:49:20 +00:00
b6f89df060
- Propagate the latest commit version of a storage server as part of read request.
...
Make storage server read at the specified version.
2021-07-15 19:48:42 +00:00
4f853b19a6
More const-correctness improvements for Reference<AsyncVar<T>> objects
2021-07-12 21:28:38 -07:00
15fe00927a
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/StorageServerInterface.h
# fdbclient/SystemData.cpp
# fdbclient/SystemData.h
# fdbserver/storageserver.actor.cpp
# flow/ProtocolVersion.h
2021-07-12 19:10:14 -07:00
4059215180
Value should not be present for ENABLE_RUN_LOOP_PROFILING
...
This fixes an unintentional change from
79ff07a071
2021-07-12 16:07:56 -07:00
77cd95473d
Merge remote-tracking branch 'origin/master' into const-serverdbinfo
2021-07-12 11:00:22 -07:00
02f243c821
Merge pull request #5110 from sfc-gh-jslocum/tss_streaming_master
...
Added Streaming Read comparison for TSS
2021-07-12 10:44:15 -07:00
6c1d913ab8
Prevent masterServer from modifying db
2021-07-11 21:11:21 -07:00
baceacacf4
- Maintain a cache of the version vector on both the Grv proxy and
...
the client. Update these caches on the (transaction) read path.
2021-07-07 19:35:26 +00:00
ccffd6f47f
Fix option validation for DISABLE_CLIENT_STATISTICS_LOGGING
2021-07-04 19:06:39 -07:00
8cc40e3a2b
Expand use of BOOLEAN_PARAM
2021-07-02 21:41:50 -07:00
79ff07a071
Added *BOOLEAN_PARAM macros to enforce documentation of boolean parameters
2021-07-02 15:04:42 -07:00
9f1afbb709
Added Streaming Read comparison for TSS
2021-07-01 19:00:05 -05:00
ab0ab765c8
Merge pull request #4965 from sfc-gh-jslocum/tss_mapping_changes
...
TSS Mismatch Changes
2021-06-29 08:24:25 -07:00
73752f441b
exclude locality:clang-format, ranged loops, documentation, tracking addStoragesever for exclusion.
2021-06-23 18:03:27 -07:00
62355571d0
exclude servers based on locality match
2021-06-23 18:03:27 -07:00
244e8f3527
Merge branch 'master' of https://github.com/apple/foundationdb into feature-streaming-reply-master
...
# Conflicts:
# fdbclient/CMakeLists.txt
# fdbclient/Knobs.h
# fdbserver/Knobs.h
2021-06-21 16:35:21 -07:00
1cec223434
cherry pick streaming replies to master
2021-06-19 09:47:13 -07:00
2d248a5926
Collect actors waiting on disk and network
2021-06-17 16:50:34 -07:00
826e70916c
Merge remote-tracking branch 'origin/master' into config-db
2021-06-17 09:47:41 -07:00
805c4200ce
Add collector to get running actor name
2021-06-16 18:08:51 -07:00
fe936207a9
Replace lower priority txn request when limit is hit.
2021-06-15 14:00:06 -07:00
0ff041222a
Add removals back
2021-06-13 18:23:59 -07:00
112be7a763
Remove most work
2021-06-13 16:19:02 -07:00
56dadaa428
TSS Mismatch Changes
2021-06-11 23:13:16 +00:00
41c790b299
Merge remote-tracking branch 'origin/master' into config-db
2021-06-10 22:31:23 -07:00
c272304e60
Manage global flow knobs with global knob collection
2021-06-09 22:33:00 -07:00
c6d96c4124
Added TSS Quarantine and fixed TSS mapping bug in CommitProxy
2021-06-09 14:42:50 +00:00
838d847d4e
Merge pull request #4860 from sfc-gh-xwang/ppwtest
...
implement perpetual storage wiggling feature
2021-06-04 16:18:39 -07:00
486a04659f
Lazy inititialization
2021-06-04 15:01:18 -07:00
9b36f69b8d
Merge pull request #4892 from sfc-gh-jslocum/tss_mappingv2
...
TSS Mapping Change
2021-06-04 14:57:51 -07:00
153de33f57
Revert "Merge pull request #4802 from sfc-gh-ljoswiak/revert/actor-lineage"
...
This reverts commit 6499fa178e1512631957
2021-06-04 13:31:55 -07:00
ba25b95c6a
Fix global config not updating on server processes
2021-06-03 14:27:24 -07:00
365b0bc4d8
better api
2021-06-03 19:57:44 +00:00
89dadf1475
Create IKnobCollection interface
2021-06-02 23:40:52 -07:00
351325b3af
comment modification; wait perpetual wiggling close
2021-06-03 05:13:20 +00:00
b3e4f182ef
TSS Mapping Change
2021-06-02 17:30:09 +00:00
4c059670a0
Parse configuration knob values on the client
2021-06-02 10:04:46 -07:00
a775f92fca
Merge remote-tracking branch 'origin/master' into config-db
2021-06-01 15:39:34 -07:00
8b9c8b33fc
manually merge with master
2021-06-01 17:51:42 +00:00
d67184163b
Merge pull request #4556 from sfc-gh-jslocum/tss
...
Testing Storage Server
2021-06-01 09:11:10 -07:00
750901dd1d
Reduce the frequency that buggified reads are failed so that transactions with a lot of reads aren't doomed to almost always fail.
2021-05-27 21:54:59 -07:00
24c0c3361a
change quietdatabase
2021-05-25 21:20:24 +00:00
4257ac2b4d
More TSS Changes/Fixes
2021-05-25 20:37:48 +00:00
ce82c9653e
Testing Storage Server implementation
2021-05-25 20:28:50 +00:00
6165af81fa
Merge branch 'apple:master' into feature-range-feed
2021-05-21 15:21:34 -07:00
bb61a20851
Improve const-correctness of ISingleThreadTransaction
2021-05-15 00:23:21 -07:00
4ea760b2a9
Revert "Merge pull request #4136 from sfc-gh-mpilman/features/actor-lineage"
...
This reverts commit da41534618e6300905d6
2021-05-10 20:26:12 -07:00
331dd2e377
Replace config_transaction option with use_config_database
2021-05-10 13:18:31 -07:00
cdf98f987d
Revert "Fix global config not triggering changes on server processes"
2021-05-10 12:18:28 -07:00
71ba6f4501
Switch std::function to AsyncTrigger
2021-05-07 16:05:03 -07:00
0fc1552683
Fix global config updates not triggering on fdbserver processes
2021-05-06 15:46:34 -07:00
5c2d7b6080
Create RangeResult type alias
2021-05-03 13:14:16 -07:00
8dcd779fc4
Merge branch 'master' into features/actor-lineage
2021-05-02 14:11:42 -07:00
2621c7153a
clang format PR
2021-04-30 10:41:35 -07:00
d652287bae
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbclient/DatabaseContext.h
# fdbclient/NativeAPI.actor.cpp
# fdbclient/StorageServerInterface.h
# fdbserver/storageserver.actor.cpp
2021-04-30 10:29:42 -07:00
f513543305
fix special keyspace register
2021-04-28 16:35:09 -06:00
904a39e473
Merge pull request #4667 from sfc-gh-ajbeamon/feature-mvc-monitor-protocol-version
...
Use fewer connections in the multi-version client
2021-04-28 14:13:17 -07:00
f5b0008346
Merge branch 'features/actor-lineage' into features/actor-lineage-fluentd
2021-04-28 09:31:20 -06:00
54919d4f3b
Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage
2021-04-28 09:22:14 -06:00
4fab2ecd30
Merge remote-tracking branch 'origin/master' into features/actor-lineage
2021-04-28 09:20:54 -06:00
2d6fafde64
Implemented configuration
2021-04-27 10:26:42 -06:00
823873a9aa
Address review comments:
...
Use nullptr instead of NULL
Use const& for a parameter
Add some comments
2021-04-26 14:39:27 -07:00
a794fca932
Support 5.0 (and earlier) client versions by adding GRV probing for old versions. Update the C bindings implementation of get_server_protocol to convert the ProtocolVersion object into a uint64_t. Rename a misleading protocol version alias.
2021-04-23 15:00:21 -07:00
52bba82e8e
Add window size configuration key
2021-04-23 14:05:05 -07:00
a6a8a97e1f
Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage-instrument
2021-04-22 17:50:36 -06:00
15336ca274
Add callback for specific global configuration key changes
2021-04-20 17:51:38 -07:00
eaaae2e16d
Merge branch master into 'feature-mvc-monitor-protocol-version'
2021-04-20 15:07:02 -07:00
d76b32da18
Annotate read paths on the server side
2021-04-20 15:10:01 -06:00
b34825a0e6
Merge branch 'master' of github.com:apple/foundationdb into add-dd-and-maintenance
2021-04-19 14:52:10 -07:00
51ce278d2d
Merge branch 'features/actor-lineage' into features/actor-lineage-collector
2021-04-19 12:55:29 -06:00
09ddcb3bae
remove old sample thread
2021-04-19 11:55:35 -06:00
7307750e5e
Merge remote-tracking branch 'origin/master' into features/actor-lineage
2021-04-19 11:29:52 -06:00
c8e9e2fab4
Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage-collector
2021-04-16 17:33:40 -06:00
551268b0f2
Add well known endpoint for worker communication
2021-04-15 13:50:50 -07:00
486260e944
Fix infinite loop with stable interface protocol monitoring. Fix case where getting an error with a network option didn't properly terminate the database connection. Reduce option lock critical section.
2021-04-15 13:36:31 -07:00
b2d6930103
The multi-version client monitors the cluster's protocol version and only activates the client library that can connect.
2021-04-15 11:45:14 -07:00
d51f94f16e
Merge pull request #4330 from sfc-gh-ljoswiak/features/global-configuration
...
Add global configuration framework implementation
2021-04-14 16:10:08 -06:00
3ed0d614d2
Move fdb_get_server_protocol to be a function on the database object. Add an argument for expected_version that can be used to signal that the function shouldn't return unless the protocol version is different.
2021-04-14 12:50:30 -07:00
7de23918c0
Add comments, fix erase bug, make optimizations
2021-04-14 10:56:33 -07:00
6de28dd916
clang-format
2021-04-14 10:56:33 -07:00
4a799baa1d
Add clear range for global configuration
2021-04-14 10:56:32 -07:00
c3f68831af
Move existing ClientDBInfo variables to global configuration
2021-04-14 10:56:32 -07:00
f1415412f1
Add global configuration framework implementation
2021-04-14 10:56:32 -07:00
eab468fecc
Remove extra line caused by commit issue
2021-04-14 09:32:48 -07:00
1d362d8869
Merge remote-tracking branch 'origin/master' into features/actor-lineage
2021-04-14 09:55:04 -06:00
feede1d2f6
Fix line length of test macro + comments to be within the 120 character limit
2021-04-13 10:48:52 -07:00
ed8d43cb87
Merge pull request #5 from sfc-gh-ljoswiak/features/network-actors
...
Sample actors waiting on network
2021-04-09 15:07:53 -06:00
5c79d29140
Use object lifetimes instead of function calls
2021-04-07 10:59:45 -07:00
c481ba2cfa
Update annotation class name
2021-04-06 17:32:02 -07:00
433872e17d
Sample actors waiting on network
2021-04-06 17:28:28 -07:00
747fa2a353
Merge branch 'master' of github.com:apple/foundationdb into add-dd-and-maintenance
2021-03-26 12:22:37 -07:00
0eff74f205
Add special keys for maintenance and datadistribution
2021-03-26 12:19:33 -07:00
de1c354b12
address pr comments
2021-03-23 20:56:37 +00:00
73571ad52e
Merge branch 'master' of github.com:sfc-gh-nwijetunga/foundationdb into network_busyness
...
* 'master' of github.com:sfc-gh-nwijetunga/foundationdb: (55 commits)
Port lost changes from #4004
Fix markdown
Factor our refreshTransaction gadget
Align FastAllocator memory to 4096 for size >= 4096
Fix heap-use-after-free's
Disable lsan in joshua
Manually apply changes made to BackupContainer.actor.cpp on release-6.3 to the new files where that code is located.
add release branch instructions
Fix PR number in release note link
fixed a simulation bug where a process on an unreliable machine would be considered reliable by the simulator
Add tini as PID 1 to docker image (#4363 )
Fix typo
Fix broken_promise bug
Explain FastAllocator<4096> special case
Inform LSAN that we have unaligned pointers
apply clang-format to flow/Platform.h
Fix typos
Address review comments
Change the macro that forbids exit() calls to be a static assertion
Add versionstamp serialization information
...
2021-03-18 18:58:43 +00:00
822cbf895d
address pr comments
2021-03-16 23:29:02 +00:00
bf5f83d2bf
address pr comments
2021-03-16 21:49:36 +00:00
5f079904d4
add unit test
2021-03-16 18:30:39 +00:00
17f9da6962
remove traces
2021-03-16 16:47:50 +00:00
191230a7b3
Factor our refreshTransaction gadget
2021-03-16 00:51:20 +00:00
f0d79b3d86
Inital implementation of network busyness
2021-03-15 23:23:56 +00:00
d45481decc
Fix broken_promise bug
2021-03-13 01:16:27 +00:00
6cacf3e002
Fix potential heap-use-after-free
...
If we destroy tr here then we might not be able to dereference |self|.
Keep the window where |self| might have been deleted small.
2021-03-12 00:26:26 +00:00
a76c7b9754
Fix several memory leaks and a thread leak
2021-03-11 23:58:32 +00:00
2c316db94d
Merge remote-tracking branch 'apple/master' into master-format-final
...
Also format the changes
2021-03-11 12:54:26 -08:00
be980ebb55
Merge pull request #4354 from sfc-gh-clin/add-advanceversion
...
Add advanceversion into special keys
2021-03-11 11:43:53 -08:00
d7252da951
clang-format: Fix the TEST() macros which require comments in line
2021-03-10 16:50:53 -08:00
7404a29d58
Merge branch 'master' of github.com:apple/foundationdb into add-dd-and-maintenance
2021-03-10 11:47:39 -08:00
df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
6cfba6e54b
added support for popping rangefeeds
2021-03-05 11:46:33 -08:00
21822e4949
more fixes
2021-03-04 19:46:32 -08:00
adaa67df63
fix registerRangeFeedActor
2021-03-04 19:42:36 -08:00
bc757f3e4b
basic framework for range feed support
2021-03-04 19:14:54 -08:00
79cec09255
Apply clang-tidy's performance-inefficient-vector-operation fix
...
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.
$ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
2021-03-04 03:58:25 +00:00
9ce9cba0e5
Merge branch 'master' of github.com:apple/foundationdb into add-advanceversion
2021-02-19 14:29:08 -08:00
703107332a
Add special keys for profile client get/set
2021-02-19 12:22:00 -08:00
9fb396e372
Merge pull request #4255 from sfc-gh-clin/add-coordinators-into-special-keys
...
Add coordinators into special keys
2021-02-18 18:26:07 -08:00
fa3834f115
fix bug
2021-02-18 18:23:28 -05:00
ac860b3c22
Add advanceversion into special keys
2021-02-18 01:27:14 -08:00
c075415a51
Merge pull request #4344 from sfc-gh-ljoswiak/fixes/remove-async-tracer
...
Remove async tracer
2021-02-17 09:43:24 -08:00
f3a849de10
Update ChangeConfig test for auto coordinators change
2021-02-16 23:55:58 -08:00
30d2d9ff82
Remove async tracer
2021-02-16 16:36:33 -08:00
82cda5448d
Merge pull request #4266 from sfc-gh-nwijetunga/optimize-watches-client
...
Optimize Watches Client
2021-02-16 13:28:51 -07:00
a59f07e6fe
address pr comment
2021-02-16 13:32:03 -05:00
13e4f6d7e1
small formatting fix
2021-02-12 12:57:33 -05:00
058557a3b1
fix pr issues
2021-02-11 15:51:37 -05:00
270a18f089
formatting fixes
2021-02-10 13:19:59 -05:00
7acfeda955
cleanup
2021-02-09 20:57:16 -05:00
386e3b0838
minor change
2021-02-09 18:36:45 -05:00
4982679ef2
change function name
2021-02-09 18:06:36 -05:00
3b48942f9a
resolve pr comments
2021-02-09 17:56:30 -05:00
d2233ffa5a
whitespace change
2021-02-09 13:38:24 -05:00
82e9f54938
small change
2021-02-09 13:36:44 -05:00
2d46cfcb80
remove traces
2021-02-09 13:33:19 -05:00
2560dc0227
add test workload
2021-02-09 12:32:37 -05:00
90686a95a0
format code
2021-02-08 22:17:16 -08:00
61082e70e1
format code, add comments
2021-02-08 19:11:30 -08:00
83af53b1bb
correctness fixes
2021-02-05 19:23:37 -05:00
a85458dd1e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-snapshot
2021-02-05 14:14:52 -08:00
97925de9d4
Update fdb_database_create_snapshot to take a UID as input
2021-02-05 14:14:38 -08:00
6c687067a6
refactor
2021-02-04 12:51:53 -05:00
ec6226bde9
change logic
2021-02-03 20:01:26 -05:00
b6e7961b78
change to references
2021-02-02 20:29:03 -05:00
c67aac8212
remove the use of pointers
2021-02-02 13:26:12 -05:00
6f38105e48
fix issue with case 4
2021-02-01 19:07:14 -05:00
6bee8ac0aa
Inital implementation
2021-02-01 18:42:13 -05:00
c2aedb0b94
clang-format, remove comments
2021-01-29 18:20:09 -08:00
7d32045506
Merge branch 'master' of github.com:apple/foundationdb into add-coordinators-into-special-keys
2021-01-29 11:46:01 -08:00
e1182f2f86
Add coordinators to special keys
2021-01-29 11:45:52 -08:00
5b5dd0a0d4
Merge pull request #4204 from sfc-gh-ljoswiak/features/network-tracer
...
Add network option to set tracer in fdbclient
2021-01-27 20:00:49 -08:00
5f912f0deb
Add snapshot c function
2021-01-26 11:53:35 -08:00
7558cd8e06
Merge remote-tracking branch 'origin/master' into simplify-global-knobs
2021-01-26 10:00:25 -08:00
a51de2f73d
Merge pull request #4220 from sfc-gh-clin/add-c-function-force-recovery
...
Add c function fdb_database_force_recovery_with_data_loss
2021-01-25 18:22:41 -08:00
29c0e05ea6
Clarify naming of distributed tracing options
2021-01-22 17:30:33 -08:00
500eabf6dd
Add network option to set tracer in fdbclient
2021-01-21 12:04:28 -08:00
fb9b4b7626
Merge remote-tracking branch 'origin/master' into misc-changes
2021-01-21 10:45:50 -08:00
ad99bf0471
Merge remote-tracking branch 'origin' into misc-changes
2021-01-21 10:03:07 -08:00
9cd7c43a2c
Add c function fdb_database_force_recovery_with_data_loss
2021-01-21 00:42:23 -08:00
fd20011276
Remove tags
2021-01-20 17:05:58 -08:00
4d6fbf099c
Use StringRef for span tags
2021-01-20 13:57:36 -08:00
7ae5e98c5a
Add metadata to a few spans
2021-01-20 13:57:36 -08:00
b0f61fb74f
Resolve the simple-looking conflicts
2021-01-15 19:35:10 +00:00
ff7d306b09
Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
...
Include conflict markers for now. Will resolve.
2021-01-15 18:04:09 +00:00
8ce9bd4e8e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2021-01-14 17:31:26 -08:00
e32a0fb742
Resolve conflicts
2021-01-14 20:13:29 +00:00
6d6b2843da
Merge remote-tracking branch 'upstream/release-6.2' into anoyes/merge-release-6.2
...
I'm including the conflict markers for now and I'll resolve them in a
subsequent commit
2021-01-14 17:20:11 +00:00
87b4463305
Solve comments: update c api description; refine code
2021-01-13 14:56:41 -08:00
8ff14878fe
Merge remote-tracking branch 'origin/master' into simplify-global-knobs
2021-01-13 14:39:35 -08:00
58360a192f
Add a counter for the number of status requests made by a client
2021-01-05 10:18:43 -08:00
e097c98956
Remove headed added by IDE, add comments
2021-01-04 00:18:01 -08:00
da7b65cd71
Change the interface back to return int64_t
2021-01-03 22:45:09 -08:00
5b2e88b187
Use structured bindings in for loops
2020-12-27 01:46:20 -04:00
0ecfaf138e
Merge pull request #4150 from sfc-gh-ljoswiak/fixes/sps-tracing
...
Remove "a/" from tracing keys in special-key-space
2020-12-21 13:48:01 -08:00
9db9a84798
Ignore negative duration value
2020-12-21 13:29:16 -08:00
b68d84aaea
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-12-21 12:32:31 -08:00
7b4f7ce4e0
update comments
2020-12-20 22:47:55 -08:00
9601769b01
Merge pull request #3858 from sfc-gh-rchen/stable_interfaces
...
Stable interfaces
2020-12-11 09:34:27 -08:00
5f57d72a59
remove print statements and format protocol version workload
2020-12-11 04:46:20 +00:00
4cb821cd63
Merge remote-tracking branch 'upstream/release-6.2' into merge-6.2-to-6.3
2020-12-09 15:47:44 -08:00
43fd545f06
Remove a/ from tracing path in special-key-space
2020-12-09 13:40:16 -08:00
9581bc0279
Merge pull request #4129 from sfc-gh-anoyes/anoyes/merge-release-6.3-master
...
Merge release 6.3 master
2020-12-08 08:57:54 -08:00
54237f28ea
Throw error on commit for late trace options
2020-12-07 14:43:44 -08:00
18201d6f4f
Add option to enable transaction tracing
2020-12-07 11:24:14 -08:00
75836d6114
Fix double counting of range reads in TransactionMetrics
2020-12-07 09:01:19 -08:00
7fbc4d7391
Resolve conflicts
2020-12-04 23:58:42 +00:00
76142549e0
Return keys from range read in lexicographic order
2020-12-04 11:21:39 -08:00
877997632d
Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
...
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
3669615e4b
Refactor and clean up
2020-12-03 14:06:11 -08:00
88e66545bb
Change the api to return bool instead of int
2020-12-03 12:01:32 -08:00
c77d9e4abe
merge conflicts
2020-12-02 21:53:19 +00:00
1f166bc183
Add options to set custom transaction ID and disable tracing
2020-11-30 14:57:17 -08:00
1f541f02be
Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge
...
Merge, leaving conflict markers for now
2020-11-24 16:55:34 +00:00
2fae7333c0
Add duration parameter in the api
2020-11-23 14:55:29 -08:00
b1e3478267
Simplified global knobs
2020-11-21 13:27:48 -08:00
2320f970c1
Add reboot_worker and pythong binding
2020-11-19 16:43:39 -08:00
aec0398fd0
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-11-17 11:39:12 -08:00
45c9a0abc7
Revert "Revert "Add limiting health metrics""
...
This reverts commit 209ebcc595
2020-11-13 17:24:57 -08:00
209ebcc595
Revert "Add limiting health metrics"
2020-11-13 17:08:46 -08:00
8c0b4dbe4c
Merge pull request #4067 from sfc-gh-tclinkenbeard/add-limiting-health-metrics
...
Add limiting health metrics
2020-11-13 16:04:44 -08:00
6c4493166f
Add limiting storage queue and durability lag to health metrics
2020-11-12 20:14:41 -08:00
0f205620bf
Merge branch 'master' into features/udp
2020-11-12 13:51:17 -07:00
1343f40117
don't allow empty coments
2020-11-11 14:07:54 -07:00
4669f837fa
Add uses of makeReference
2020-11-07 22:10:18 -08:00
4788544a6f
Revise comments based on review suggestions
...
Ack. Jingyu and Xin for their suggestions.
2020-11-06 08:51:13 -08:00
063700e4d6
Add comments and questions to HA and tLog code reading
...
The comments' correctness need to be confirmed by reviewers.
2020-10-30 12:14:57 -07:00
cf4c8e375f
Merge remote-tracking branch 'origin/release-6.3' into merge
2020-10-29 22:15:41 -07:00
a72bb52eae
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2020-10-28 22:32:44 -07:00
bf7bf470d9
Merge branch 'master' of https://github.com/apple/foundationdb into stable_interfaces
2020-10-26 19:36:19 +00:00
a15bc32b14
Merge pull request #3945 from sfc-gh-clin/add-consistencycheck-to-special-keys
...
Add consistencycheck command to special keys
2020-10-26 10:59:17 -07:00
309e38dc06
add unit test and some cleanup
2020-10-23 19:34:18 +00:00
055add9682
conflicts
2020-10-23 06:33:00 +00:00
90caa24af1
add function to get correct checksum class. Remove IChecksum::checkSum function
2020-10-23 02:24:29 +00:00
c8eb003ec4
Change to \xff\xff/management/in_progress_exclusion, the snake case naming style
2020-10-22 18:05:54 -07:00
c8b3b37b4a
Remove unused code for #3930
2020-10-22 17:04:08 -07:00
95654ce7e7
Change dbLocked to snake_case naming db_locked
2020-10-22 11:35:39 -07:00
2b4ec39a2f
Avoid passing references from an iterator into an insertion that invalidates the iterator.
2020-10-22 11:30:08 -07:00
1d64b2c335
clang-format
2020-10-22 11:28:03 -07:00
b728fe473e
Add consistencycheck command to special keys
2020-10-22 11:08:54 -07:00
8d937a56f2
Throw operation_failed() if dd not found when calling ddMetricsRequestServer
...
Remove time_out error in DDMetricsList rpc
Add retry logic for dd special key when dd not found
Update FuzzApiCorrectness test
2020-10-21 15:28:03 -07:00
8cc3e4d3c6
Merge release-6.3 into master
2020-10-19 22:51:56 -07:00
c84a5f4014
change IChecksum interface and remove coordinator protocol version optional
2020-10-19 21:39:07 +00:00
a4ccf8c5b6
add checksum class to change checksum in the future. expose get server protocol api
2020-10-19 16:57:11 +00:00
24ea35e56f
Merge pull request #3748 from sfc-gh-ljoswiak/visibility-2
...
Add TLogVersion::V6
2020-10-14 17:35:32 -07:00
bbc6d2aa03
Merge pull request #3872 from sfc-gh-clin:add-lock-unlock-to-special-keys
...
Add lock/unlock to special keys
2020-10-13 09:09:49 -07:00
5d902f9177
Merge pull request #3394 from dongxinEric/feature/range-split-points-based-on-size
...
Feature/range split points based on size
2020-10-08 14:25:11 -07:00
c66a775d64
Merge remote-tracking branch 'upstream/master' into add-lock-unlock-to-special-keys
2020-10-08 14:23:16 -07:00
53d3759e73
Add lock command to special key \xff\xff/dbLocked
2020-10-08 14:23:02 -07:00
f03a355c4e
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/downloads.rst
# fdbrpc/FlowTransport.actor.cpp
# fdbrpc/FlowTransport.h
# flow/Arena.h
# flow/Knobs.cpp
# flow/Knobs.h
# packaging/msi/FDBInstaller.wxs
2020-10-07 12:28:22 -07:00
dea7000970
Merge remote-tracking branch 'upstream/master' into visibility-1
2020-10-06 18:38:15 -07:00
76da0a7cbc
Backport #3426 to release-6.2
2020-10-05 13:53:51 -07:00
b294e148c5
another compile fix
2020-10-04 16:54:43 -07:00
614c8bc895
Get read versions requests must be load balanced on the number of requests because ratekeeper gives out an equal budget to each proxy
2020-10-04 16:20:24 -07:00
bbc7ce581e
Resolve conflicts merging from 6.3 to master
2020-09-25 16:08:31 -07:00
862336de8f
Merge branch 'master' into mengxu/merge-to-master-PR
2020-09-24 17:06:00 -07:00
b37daf88d4
add jitter to the delay
2020-09-22 22:24:40 -07:00
09ff05c629
backed out buggify of apiVersion because there are too many tests that depend on it for a patch release
2020-09-22 21:15:49 -07:00
e0fbadfeb5
Added a buggify so older API versions are run in simulation
2020-09-22 18:58:01 -07:00
c99627a7ab
Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-apiverison-batch-error
2020-09-22 17:06:17 -07:00
7dec47938e
protect api versions older than 630 from seeing batch_transaction_throttled errors
2020-09-22 17:05:09 -07:00
b4e209bd4e
Replace NULL with nullptr in fdbclient
2020-09-20 11:33:09 -07:00
a52ef0acb8
Merge pull request #3677 from sfc-gh-clin/add-setclass-to-special-key-space
...
Add setclass to special keys
2020-09-19 14:55:03 -07:00
71309a79f9
Merge branch 'release-6.3' into range
2020-09-17 17:54:33 -07:00
cc5bc16bd8
Rename more places from proxy to commit proxy
2020-09-15 22:29:49 -07:00
3dfadff6d8
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-09-14 16:45:47 -07:00
cc32074ed7
Rename actor
2020-09-14 16:45:34 -07:00
35bef73a1c
Rename proxy to commit proxy
2020-09-10 17:44:15 -07:00
d270b15c89
Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space
2020-09-09 16:21:19 -07:00
1ca7fe1a05
Add span metadata message
2020-09-04 15:36:47 -07:00
ae7bf24353
Merge pull request #3549 from yliucode/grv-proxy
...
Separate out a new role GrvProxy to serve GRVs.
2020-09-03 19:03:45 -07:00
f9624b5a1a
Avoid unnecessary copy
2020-09-03 13:56:23 -07:00
b4c96cadc7
Merge branch 'release-6.3' into merge-release-6.3-into-master
2020-09-02 12:45:57 -07:00
93da4957ad
Update fdbclient/NativeAPI.actor.cpp
2020-09-01 13:17:01 -07:00
800ece286d
Results from read hot range requests were not held by the results arena.
2020-09-01 13:14:27 -07:00
fd7198d874
Extend backup container interface to support query restorable files set by key ranges
2020-08-29 19:58:07 -07:00
e87327b33b
Merge master branch and keep master proxy reporting txn cost estimation to ratekeeper
2020-08-29 12:47:35 -07:00
cd86ca6850
Merge branch 'master' into feature/range-split-points-based-on-size
2020-08-27 10:06:02 -07:00
e8e24a8ed6
Resolve comments
2020-08-26 21:04:35 -07:00
a07b9f234d
Add readonly range \xff\xff/configuration/process/class_source, and change \xff\xff/configuration/class/ to \xff\xff/configuration/process/class_type/
2020-08-25 18:18:32 -07:00
63b3612ad5
Merge master branch and resolve conflicts
2020-08-24 16:42:31 -07:00
3afdb44c7a
merge master
2020-08-23 17:09:04 +00:00
b4cec6577a
Fix merge bugs
2020-08-22 15:04:42 -07:00
e1b7dd0c7d
Merge remote-tracking branch 'upstream/release-6.3' into dyoungworth/fixMerge1
2020-08-22 12:25:19 -07:00
23ef23357c
Test code to add c function for kill/suspend
2020-08-21 14:30:52 -07:00
9b096ab58e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2020-08-21 12:20:35 -07:00
3b63d8b01b
remove FIXME; remote tagSet.reset(); trivial changes
2020-08-21 19:17:16 +00:00
5aaf86206a
add counter for expensive clear estimation for audit and trace
2020-08-20 23:00:45 +00:00
556b239057
clang-format
2020-08-20 14:51:41 -07:00
bff20faa42
Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space
2020-08-20 13:50:54 -07:00
e375761581
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-20 20:29:14 +00:00
9c709b33d6
scale cost
2020-08-19 19:47:41 +00:00
599675cba8
modify some details to get better performance
2020-08-19 04:23:23 +00:00
7b210aee24
Use =delete to disable special member functions
...
This has the benefits of:
- Easier to understand compiler error messages
- Moves some errors from link-time to compile-time
- NonCopyable classes can still specify default move constructors
2020-08-18 11:07:00 -07:00
24dbb3961c
Knob adjustment
2020-08-17 22:38:22 +00:00
130bd7ac22
Fixed a bug that would potentially cause invalid memory access, i.e. segmentation fault.
2020-08-17 13:16:17 -07:00
daead84fdd
Merge master
2020-08-14 14:07:08 -07:00
548ba06510
change map to deque; code style problem
2020-08-14 18:30:58 +00:00
475d4cca72
solve some cr comments
2020-08-14 16:40:16 +00:00
b9474055a5
Add \xff\xff/configuration/class to read process class
2020-08-14 01:01:08 -07:00
e94de126de
merge with master
2020-08-14 01:59:09 +00:00
20700f75f4
Merge master
2020-08-13 14:28:27 -07:00
b2bba8ccbb
Correct fix for debug ID logging
2020-08-13 13:37:15 -04:00
002cca5e58
Merge branch 'master' into tmp/merge-PR
...
Conflicts:
fdbclient/NativeAPI.actor.cpp
fdbserver/StorageMetrics.actor.h
2020-08-12 20:08:13 -07:00
8bc57c16f0
Merge pull request #3637 from dongxinEric/misc/fix-spammy-readhot-logging
...
Increased some knobs to throttle the spammy read hot logging.
2020-08-12 16:46:32 -07:00
8f1deaa47e
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-12 23:32:23 +00:00
3ac6996844
Increased some knobs to throttle the spammy read hot logging. Also added more details inside the read hot log to make it useful
2020-08-12 14:27:34 -07:00
cc1059be75
Make logging of server trace ID consistent with other code
2020-08-12 19:05:01 +00:00
f3ecf14601
change midShardSize type and other details
2020-08-12 17:49:12 +00:00
16ef2bd3bd
Pending commit
2020-08-12 10:34:07 -07:00
79ce16650d
merge master branch
2020-08-11 19:22:10 -07:00
4782c7c4b1
Merge remote-tracking branch 'origin/master' into features/udp
2020-08-11 15:35:52 -06:00
b66dab8816
Fix span early destruction
...
We want the span to stick around for the full duration of
extractReadVersion since it writes to the log from its destructor.
2020-08-11 11:19:04 -07:00
c0b7ac0b7d
try to throttle write tag and read tag seperatedly
2020-08-10 23:43:58 +00:00
cae6f04070
sample on cost instead of bytes
2020-08-10 22:30:06 +00:00
696e77c94e
query midShardSize from proxy
2020-08-10 20:13:44 +00:00
df9149fea4
ignore transaction tag of immediate transactions
2020-08-07 23:36:17 +00:00
98c39ad9d4
Fix the build error.
2020-08-07 14:03:42 -07:00
a46e620baa
Merge branch 'master' into feature/range-split-points-based-on-size
2020-08-06 15:23:11 -07:00
6d84bcb568
Merge remote-tracking branch 'origin/master' into features/udp
2020-08-06 14:08:34 -06:00
8976694ba1
UDP implementation (untested)
2020-08-06 14:06:50 -06:00
ba66b1f668
trivial changes
2020-08-06 14:53:26 +00:00
d6a23a4d6b
Resolve comments to make GRV proxy a separate process class
2020-08-06 00:01:57 -07:00
13307679c5
use median shard size"
2020-08-05 03:57:25 +00:00
4997754cf8
fix format in monitorDDMetricsChanges
2020-08-04 04:14:11 +00:00
00f2c38e3a
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-08-03 20:55:21 -07:00
48094ea17e
add INCOMPLETE_SHARD_PLUS knob
2020-08-04 03:18:05 +00:00
52073738c0
fix bugs; change some KNOB values
2020-08-03 22:18:34 +00:00
b903e60cb7
fix monitorDDMetricsChanges bugs
2020-08-03 17:12:36 +00:00
6db7040f17
update DDMetrics periodically
2020-08-03 05:52:38 +00:00
d1cc87452c
merge with master; solve conflicts; solve initialization;
2020-08-02 22:44:07 +00:00
92c1112c74
consider clear single key
2020-08-01 18:20:13 +00:00
4f7dab4951
sample clear op on client
2020-08-01 06:14:52 +00:00
fc3dfc170f
Merge branch 'master' of https://github.com/apple/foundationdb
2020-07-31 19:09:33 +00:00
c3a629588f
add client transaction tag sample
2020-07-31 19:08:42 +00:00
a048e5e159
Change prefix to \xff\xff/management and make decode/encode as pure functions
2020-07-31 02:18:49 -07:00
f2662a9a74
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-30 19:01:11 -07:00
10540a3c73
Change error name and fix a bug in ExclusionInProgressRangeImpl::getRange()
2020-07-30 18:58:09 -07:00
11cbcffc9a
Temporary code
2020-07-30 16:07:54 -07:00
b5dc6b715f
Add file comment for GrvProxyServer.actor.cpp
2020-07-29 12:10:53 -07:00
30ea639666
Remove debug traces
2020-07-29 07:55:05 -07:00
f7b76a92af
pass joshua
2020-07-29 07:26:55 -07:00
602b0f9dea
Merge branch 'master' into vectorref-emplace-back
2020-07-28 17:17:11 -07:00
a641429d63
Add an enum for implementation type and use for registeration
2020-07-27 12:29:03 -07:00
ec9553b6c6
change \xff\xff/failure to \xff\xff/error_message
2020-07-24 11:38:56 -07:00
525f10e30c
Merge master branch
2020-07-22 16:08:49 -07:00
302cf5c45f
Remove debug trace events
2020-07-22 12:20:22 -07:00
2703cedac5
Fixed known bugs
2020-07-17 22:24:52 -07:00
0df2a8d014
better code style
2020-07-18 01:48:58 +00:00
1c51aaeea5
Use static keys in SpecialKeySpace
2020-07-17 15:14:18 -07:00
96e5a4c827
Replace all \xff\xff/conf with managementApiRange.begin
2020-07-17 11:56:27 -07:00
5fa6785fb8
Add special keys for command options
2020-07-16 18:15:35 -07:00
9d0d189cc8
better serialize; TransactionOption::clear patch
2020-07-15 22:39:21 +00:00
eb44ae0e86
finish local shard estimation
2020-07-15 16:08:00 +00:00
5b06d69d25
Pass watches test
2020-07-15 00:37:41 -07:00
bcb858288b
eliminate cost estimation when tagSet is empty
2020-07-14 23:07:21 +00:00
3f4da830a2
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-14 10:36:47 -07:00
a310faf9d1
solve some code reviews
2020-07-14 17:19:55 +00:00
d512170cd8
add clear cost estimation
2020-07-14 00:18:52 +00:00
60d1bfc247
Add read impl for inProgressExclusionServers
2020-07-13 13:35:25 -07:00
6ec2f92a8d
merge master
2020-07-13 17:22:37 +00:00
6374cd96c9
add commit cost estimation
2020-07-10 23:07:29 +00:00
2e5ae8eed5
Change iterator name to small case
2020-07-10 10:56:41 -07:00
cc60597739
Merge branch 'master' into bugfixes/visibility-perf-regression
2020-07-09 17:04:04 -06:00
69864c9f96
Make Spans not allocate heap memory
2020-07-09 11:49:33 -06:00
1b68665228
Merge branch 'master' into release-6.3
2020-07-08 18:52:05 -07:00
3eb3e23ae0
Merge pull request #3425 from sfc-gh-anoyes/anoyes/health-metrics
...
Expose health metrics via special keys
2020-07-08 10:37:31 -07:00
0fbe7101c3
Revert "Revert "Request tracing""
...
This reverts commit 327cc31e35
2020-07-07 10:06:13 -06:00
a43afb9d97
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-06 15:47:00 -07:00
0361eb5079
Add FailedServersImpl
2020-07-06 15:40:21 -07:00
784d0f6d00
A basic feasible version for exclude and include commands
2020-07-06 11:02:48 -07:00
bcbca81dfe
Remove ROW_LIMIT_UNLIMITED and BYTE_LIMIT_UNLIMITED from CLIENT_KNOBS
2020-07-04 12:48:44 -07:00
d689d34634
Enable range split bindings tests
2020-07-02 14:05:12 -07:00
00e9f8b9bf
Added bindings tests; Protected new SSI endpoints under new 7.0 ProtocolVersion
2020-07-02 14:05:12 -07:00
eb28492900
Fixed a bug which leads to inaccurate metrics being reported. Added tests for the new API.
2020-07-02 14:05:11 -07:00
ea08bc5462
Added server side code for range split support
2020-07-02 14:05:11 -07:00
ae63304392
Address Meng's comment
2020-07-01 20:23:18 -07:00
3ecdaf24ce
Use the actual range begin and end key instead of the shard begin and shard end key
2020-07-01 16:07:59 -07:00
409ccf3be2
Use snake_case to match status json convention
2020-06-30 17:05:29 +00:00
e1dfa410c1
Remove "Storages" field from data_distribution_stats
...
It seems like a cool idea, but it feels rushed to me. Reverting for now.
2020-06-30 15:24:16 +00:00
a55b0771c5
Test schemas
...
Also remove an ASSERT_WE_THINK that doesn't seem to hold now that we
include shards
2020-06-30 15:24:16 +00:00
f8bb76a607
Add CLIENT_BUGGIFY
2020-06-30 15:24:16 +00:00
b1d5e99494
Update documentation
2020-06-30 15:24:16 +00:00
1c30adce74
WIP
2020-06-30 15:24:16 +00:00
0db3145e82
Include storage id's in data_distribution_stats
2020-06-30 15:24:15 +00:00
e304697eca
WIP
2020-06-30 15:24:15 +00:00
ffd1467367
Add \xff\xff/metrics/health/*
2020-06-30 15:24:15 +00:00
e2c5938c28
Update module registration for readwrite module
2020-06-29 15:37:46 -07:00
1082ab2caa
fix ClientVersionRef versionString should be passed by ref
2020-06-29 21:23:56 +02:00
18ff6efbf3
Add failure module for special key space
2020-06-27 12:22:32 -07:00
f011ff2c66
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-06-23 11:23:18 -07:00
42159ccfe0
Add -Wpessimizing-move and -Wredundant-move for clang
2020-06-22 23:45:03 +00:00
59361cedaa
Rename to have read and write impls and add set, clear, clearrange interfaces
2020-06-22 14:02:03 -07:00
85a284b13c
Use VectorRef::emplace_back in several places
2020-06-19 23:02:28 -07:00
327cc31e35
Revert "Request tracing"
2020-06-16 12:32:42 -07:00
96206a8032
Merge pull request #3368 from apple/release-6.3
...
Merge Release 6.3 to master
2020-06-15 20:15:22 -07:00
fabdffd6e4
Merge pull request #3332 from sfc-gh-tclinkenbeard/fix-memaccess-warnings
...
Fix class-memaccess warnings
2020-06-15 15:22:19 -07:00
3a914c71f2
Add TransactionOptions::clear function
2020-06-13 17:17:06 -07:00
8595813b7d
Merge pull request #3355 from apple/release-6.3
...
Merge Release 6.3 into master branch
2020-06-12 20:08:47 -07:00
80334351c7
Merge pull request #3329 from sfc-gh-mpilman/features/flatbuffers-debugtx
...
Request tracing
2020-06-12 14:50:19 -07:00
ae6b716820
addressed more review comments
2020-06-11 09:58:41 -07:00
9ef93714ac
Address review comments
2020-06-10 15:48:49 -07:00
4ab3441a95
Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx
2020-06-10 15:31:29 -07:00
d9ddb84d34
remove weird include (added by clangd)
2020-06-10 13:18:26 -07:00
a332dff2ec
Fix class-memaccess warnings
2020-06-10 12:34:14 -07:00
99bf993815
Replace BOOST_NOEXCEPT with noexcept
2020-06-09 22:39:19 -07:00
c829b12bbf
defined spans for read requests
2020-06-08 17:26:15 -07:00
09c136e434
Framework for transaction tracing
2020-06-08 16:09:37 -07:00
55448a8d5a
Remove call in watch actor instead of Transaction::watch
2020-06-08 18:19:18 +00:00
731b77a8c2
Don't double count watches for too_many_watches
2020-06-07 23:54:22 +00:00
23a565ec63
Few bug fixes.
2020-06-05 16:27:04 -07:00
cf13e00a8f
Merge remote-tracking branch 'origin/release-6.3' into fdb_cache_wo_allocator
2020-06-01 17:38:31 -07:00
ba63a66d12
Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx
2020-05-29 15:38:09 -07:00
8a7d98ab21
Started instrumentation
2020-05-27 15:36:04 -07:00
86f712657e
Eliminate the undefined behavior of calling run_network twice, instead returning an error.
2020-05-22 13:31:06 -07:00
a2fd6d46a0
Cancel pending special key space reads on destroy
...
Previously the special key space would take a reference to the
transaction, but this doesn't make sense since the transaction might not
be refcounted
2020-05-21 17:37:11 +00:00
0d9135ee57
Merge remote-tracking branch 'upstream/release-6.3' into add-data-distribution-metrics
2020-05-20 15:21:26 -07:00
06676fc88f
Merge pull request #3166 from tclinken/6.3-avoid-ub
...
Eliminate some undefined behavior and const_cast usage
2020-05-20 14:36:30 -07:00
b589c3c940
Remove IncludeVersions.h, which is no longer needed for cmake-only builds
2020-05-20 13:23:02 -07:00
16990aeb6f
Add dcId to client transaction events.
2020-05-19 16:07:54 -07:00
00dce3ca1d
clang-format
2020-05-18 14:23:17 -07:00
fcca49f4d5
Add back an std::move that was accidentally removed. Some other small tweaks to avoid map lookups when tags aren't used.
2020-05-18 13:45:51 -07:00
bf62ef7c3e
Add dd-stats into special-key-space
2020-05-18 10:38:23 -07:00
6ce574f5ad
Merge remote-tracking branch 'upstream/release-6.3' into add-data-distribution-metrics
2020-05-17 23:36:52 -07:00
8597b38d10
Add an assertion in SingleSpecialKeyImpl
2020-05-15 16:34:43 -07:00
4a13107805
Add a test for special_keys_cross_module_read, fix two bugs, add several TODOs
2020-05-15 16:34:43 -07:00
c61e198e40
Using enum class instead of enum
2020-05-15 16:34:43 -07:00
e1c4e5f619
Add enums for special-key-space modules and refactor related code
2020-05-15 16:34:42 -07:00
f605c8d74f
Narrow keyrange of special-key-space to [\xff\xff, \xff\xff\xff). Initilize new ones in correctness tests
2020-05-15 16:34:42 -07:00
667b224fc8
Several changes to reduce unnecessary moves/copies
2020-05-13 19:59:34 -07:00
da14e1eb04
Fix: debug tagging was always enabled
2020-05-12 14:11:03 -07:00
8bd5dcaff8
Merge branch 'master' into atn34/special-key-versioning
2020-05-09 15:34:20 -07:00
02307ba7b6
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/DatabaseContext.h
2020-05-09 07:50:29 -07:00
Evan Tschannen