df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
aaf0a9aa7b
Merge branch 'release-6.3' into merge-release-6.3-into-master
...
# Conflicts:
# build/docker-compose.yaml
# cmake/ConfigureCompiler.cmake
# fdbclient/FileBackupAgent.actor.cpp
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/IAsyncFile.h
# fdbrpc/IRateControl.h
# fdbrpc/simulator.h
# fdbserver/KeyValueStoreSQLite.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbservice/ServiceBase.cpp
2021-02-08 12:58:34 -08:00
67e783acf8
Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
...
# Conflicts:
# cmake/CompileBoost.cmake
# cmake/FDBComponents.cmake
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/simulator.h
# fdbserver/KeyValueStoreSQLite.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/storageserver.actor.cpp
# flow/Knobs.h
# flow/network.h
2021-02-08 09:20:28 -08:00
8547abbbe4
Bug fix: In simulation, KVStoreSQLite can outlive its process during a simulated reboot so its file references must be cleared explicitly during shutdown instead of implicitly during destruction. Re-enabled a useful trace event. Some cleanup, removed unnecessary member var initializer and removed unnecessary method.
2021-02-05 21:26:39 -08:00
f6342376ab
Update the knob name
2021-02-02 13:31:00 -08:00
17671604c0
Add option to define I/O size
2021-02-02 12:39:05 -08:00
e4b953ab38
Merge pull request #4229 from sfc-gh-satherton/afcache-write-limit
...
AsyncFileCached RateControl support and bug fixes
2021-01-31 15:13:03 -07:00
d607ba3ac1
Removed merge artifact which calls functions not present in release-6.2.
2021-01-31 03:59:36 -08:00
15aa877388
Use VFSAsyncFile::checkInjectedError() to replace old injected fault detection, see pr#4212
2021-01-30 18:25:43 -08:00
bb6608ccd8
Another bug fix in SQLite file instance lifetimes. Writer can be destroyed before doClose() or stopOnError() have a chance to react, so Writer can't be used to get access to file handles to remove the rate limiter and trigger pending requests to complete. New logic is for Writer initialization to place opened file handles in KeyValueStoreSQLite instance and move disableRateControl() there, to be called from doClose().
2021-01-30 18:23:50 -08:00
4982787a80
Another attempt to fix SQLite lifetime issues. KeyValueStoreSQLite::doClose() can now access the Writer instance and call disableRateControl() on its SQLiteDB instance prior to stopping the reader and writer thread pools. This should prevent rate limiting from interfering with SQLite shutdown.
2021-01-30 18:18:36 -08:00
b0fd6bfbcf
More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent.
2021-01-30 18:18:16 -08:00
52c2695f59
Bug fixes involving use of AFCached rate control in simulation.
2021-01-30 18:13:44 -08:00
e56fe02a25
Merge branch 'add-throttling-on-AsyncfileCached' of github.com:sfc-gh-clin/foundationdb into afcache-write-limit
2021-01-30 18:11:54 -08:00
14af5857ed
Another refactor of SQLite injected error handling to address failures of the previous attempt. Some code cleanup. Added documentation of injected error handling works and why.
2021-01-30 18:06:05 -08:00
e72d486215
Refactored how injected fault state is managed to make debugging injected fault handling easier. Moved injected error tracking for open() into an INetwork global since it should be unique per simulated process. Fixed bug where injected error in WAL file was not recognized as injected because it occurred before SQLiteDB's vfsWAL pointer was initialized. Simplified logic in SQLiteDB. Added comments.
2021-01-30 18:06:04 -08:00
1a5c40fcef
Bug fixes: Check for null pager and wal pointers, and initialize vfsWAL after SQLite has opened the WAL.
2021-01-30 18:06:04 -08:00
e1f8023577
Rewrote how injected faults are handled in SQLite to be more reliable and work with an upcoming write throttling feature in AsyncFileCached.
2021-01-30 18:06:04 -08:00
4c4d1ec6b0
Use VFSAsyncFile::checkInjectedError() to replace old injected fault detection, see pr#4212
2021-01-29 15:28:08 -08:00
14e39a8747
Another bug fix in SQLite file instance lifetimes. Writer can be destroyed before doClose() or stopOnError() have a chance to react, so Writer can't be used to get access to file handles to remove the rate limiter and trigger pending requests to complete. New logic is for Writer initialization to place opened file handles in KeyValueStoreSQLite instance and move disableRateControl() there, to be called from doClose().
2021-01-28 00:59:05 -08:00
e3a6531477
Another attempt to fix SQLite lifetime issues. KeyValueStoreSQLite::doClose() can now access the Writer instance and call disableRateControl() on its SQLiteDB instance prior to stopping the reader and writer thread pools. This should prevent rate limiting from interfering with SQLite shutdown.
2021-01-27 05:31:02 -08:00
989fee908a
More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent.
2021-01-27 00:01:33 -08:00
564c70acb4
Merge branch 'master' into anoyes/modernize-use-override
2021-01-25 18:26:18 -08:00
4ee97c0784
Use clang-tidy to automatically fix missing overrides
...
Use `clang-tidy -p . $file -checks='-*,modernize-use-override' -header-filter='.*' -fix`
to fix missing overrides, and then use git clang-format to reformat just
those changes. This went pretty well for most files.
Formatting the following files went off the rails, so I'm going to
follow up with a commit that's just clang-tidy and no clang-format.
- fdbclient/DatabaseBackupAgent.actor.cpp
- fdbclient/FileBackupAgent.actor.cpp
- fdbserver/OldTLogServer_4_6.actor.cpp
- fdbmonitor/SimpleIni.h
- fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
2021-01-26 02:04:12 +00:00
adde95bea5
Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
...
Commit merge conflicts for now
2021-01-25 18:55:24 +00:00
0d32bd0c44
Bug fixes involving use of AFCached rate control in simulation.
2021-01-24 01:50:09 -08:00
523a8c08db
Merge branch 'add-throttling-on-AsyncfileCached' of github.com:sfc-gh-clin/foundationdb into afcache-write-limit
...
# Conflicts:
# flow/Knobs.h
2021-01-17 05:30:55 -08:00
11927f6fff
Another refactor of SQLite injected error handling to address failures of the previous attempt. Some code cleanup. Added documentation of injected error handling works and why.
2021-01-17 04:17:13 -08:00
f1adafaf8c
Refactored how injected fault state is managed to make debugging injected fault handling easier. Moved injected error tracking for open() into an INetwork global since it should be unique per simulated process. Fixed bug where injected error in WAL file was not recognized as injected because it occurred before SQLiteDB's vfsWAL pointer was initialized. Simplified logic in SQLiteDB. Added comments.
2021-01-16 05:04:30 -08:00
8df6874417
Bug fixes: Check for null pager and wal pointers, and initialize vfsWAL after SQLite has opened the WAL.
2021-01-15 21:13:31 -08:00
6da78b1b19
Rewrote how injected faults are handled in SQLite to be more reliable and work with an upcoming write throttling feature in AsyncFileCached.
2021-01-15 19:29:14 -08:00
0613a149a6
Add rate control on both db file and log file
2020-12-11 00:22:26 -08:00
9156d38216
Add knob SQLITE_READER_THREADS to control the number of read threads
2020-12-09 12:09:07 -08:00
5b55216252
Add rate control to AsyncFileCached
2020-12-09 12:03:32 -08:00
7fbc4d7391
Resolve conflicts
2020-12-04 23:58:42 +00:00
877997632d
Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
...
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
aa07df6a91
Backport to 6.3 the ability to read xxhash3 checksum for sqlite pages
2020-11-24 00:28:49 -08:00
4029217c2f
Update fdbserver/KeyValueStoreSQLite.actor.cpp
...
Co-authored-by: Andrew Noyes <63815641+sfc-gh-anoyes@users.noreply.github.com>
2020-11-23 10:07:08 -08:00
156a617ed8
Replace xxhash64 with xxhash3
...
The goal here is to improve performance
2020-11-21 18:25:35 -08:00
bcda617b80
Benchmark hashing algorithms in flowbench
2020-11-17 23:14:55 -08:00
e9bceda8ca
Add code coverage macros to PageChecksumCodec::checksum
2020-11-16 19:13:18 -08:00
287ab51502
Fix some bugs in PageChecksumCodec::checksum
2020-11-15 20:35:05 -08:00
145cddbb99
Clear the first 8 bits of xxHash64 checksum of SQLite pages
...
When upgrading a cluster to 7.0 which has stale data from 6.2, we don't
want to calculate an xxHash64 checksum for every page that's read. By
clearing the first 8 bits of all checksums written using xxHash64, the
server can skip this checksum in 255/256 cases when reading pages
written from version <=6.2.
2020-11-15 17:48:44 -08:00
c5689c4a72
Use xxhash64 for checksumming SQLite pages
2020-11-15 17:48:44 -08:00
4669f837fa
Add uses of makeReference
2020-11-07 22:10:18 -08:00
410d418c3e
Revert "Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining"
...
This reverts commit e258dffc650e96233015
2020-10-19 09:28:38 -07:00
dc4612712d
Merge remote-tracking branch 'origin/master' into use-override-more
2020-10-09 07:12:41 -07:00
e258dffc65
Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining
...
Pipeline the commit process in SS.
2020-10-07 20:34:57 -07:00
8571dcfe28
Use override where applicable in fdbserver
2020-10-07 18:41:19 -07:00
34d1d04904
Commit pipelining attempt #2
2020-09-23 20:03:04 -07:00
FDB Formatster