df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
79cec09255
Apply clang-tidy's performance-inefficient-vector-operation fix
...
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.
$ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
2021-03-04 03:58:25 +00:00
4ee97c0784
Use clang-tidy to automatically fix missing overrides
...
Use `clang-tidy -p . $file -checks='-*,modernize-use-override' -header-filter='.*' -fix`
to fix missing overrides, and then use git clang-format to reformat just
those changes. This went pretty well for most files.
Formatting the following files went off the rails, so I'm going to
follow up with a commit that's just clang-tidy and no clang-format.
- fdbclient/DatabaseBackupAgent.actor.cpp
- fdbclient/FileBackupAgent.actor.cpp
- fdbserver/OldTLogServer_4_6.actor.cpp
- fdbmonitor/SimpleIni.h
- fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
2021-01-26 02:04:12 +00:00
ff7d306b09
Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
...
Include conflict markers for now. Will resolve.
2021-01-15 18:04:09 +00:00
2ce967bd77
Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-redwood
2021-01-05 21:34:26 -08:00
4c28341351
Performance bug fix: Every page cache hit was calling forwardError() unnecessarily.
2021-01-05 21:34:16 -08:00
d7ebae07f5
Added yield to remap cleanup loop to prevent a stack overflow if too many queue entries are processed without waiting on IO.
2020-12-12 02:40:06 -08:00
4669f837fa
Add uses of makeReference
2020-11-07 22:10:18 -08:00
73e77f31eb
Merge pull request #3928 from yliucode/merge
...
Merge release-6.3 into master
2020-10-20 15:48:46 -07:00
8cc3e4d3c6
Merge release-6.3 into master
2020-10-19 22:51:56 -07:00
410d418c3e
Revert "Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining"
...
This reverts commit e258dffc650e96233015
2020-10-19 09:28:38 -07:00
0b46af2925
Added a simulation-only check for pager remap cleanup writing the same page twice in a cleanup cycle, which should never happen, but a bug leading to this was fixed recently. Adjusted some buggify logic to widen edge case coverage around remap cleanup parameters.
2020-10-14 21:48:03 -07:00
dc35f2b4f5
Bug fix: In page remap cleanup, if a page update's next update was at exactly the oldest retained version then the earlier update would still choose to copy the updated page over top of the original (but it shouldn't) which would race with the later update's copy if the same remap cleanup cycle pops both updates from the queue.
2020-10-13 02:26:46 -07:00
dc4612712d
Merge remote-tracking branch 'origin/master' into use-override-more
2020-10-09 07:12:41 -07:00
a9607bdcec
Explicitly seal classes that inherit but aren't inherited from
2020-10-07 21:58:24 -07:00
8571dcfe28
Use override where applicable in fdbserver
2020-10-07 18:41:19 -07:00
34d1d04904
Commit pipelining attempt #2
2020-09-23 20:03:04 -07:00
12edadd059
Merge branch 'release-6.3'
...
# Conflicts:
# CMakeLists.txt
# fdbclient/Knobs.cpp
# fdbclient/MasterProxyInterface.h
# fdbrpc/simulator.h
# fdbserver/MasterProxyServer.actor.cpp
# tests/fast/CycleAndLock.txt
# tests/fast/TxnStateStoreCycleTest.txt
# tests/fast/VersionStamp.txt
# tests/slow/ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt
# tests/slow/ParallelRestoreOldBackupCorrectnessCycle.txt
# versions.target
2020-08-31 19:33:34 -07:00
89ae0aaf39
In Redwood unit test, if the verify actor falls far behind the latest commit version it will unnecessarily verify the latest version repeatedly which extends the test's run time needlessly.
2020-08-29 23:47:54 -07:00
167d3c5a8f
To avoid running too long, the Redwood correctness unit test now limits the size of the KV verification map. This map only grows as the test runs, as it contains a record for every key set or clear committed during the test.
2020-08-29 16:44:38 -07:00
e1b7dd0c7d
Merge remote-tracking branch 'upstream/release-6.3' into dyoungworth/fixMerge1
2020-08-22 12:25:19 -07:00
d53e6cf687
Bug fix in KV byte counts.
2020-08-18 17:58:57 -07:00
090241f0df
Count BTreeCursor page reads that come from the cursor's cache of pages as cache hits in RedwoodMetrics. BTree Unit Test now verifies all data at the initial version after recovery from disk.
2020-08-15 14:30:49 -07:00
051ec7066a
BTree unit test duration now targets a number of page read/writes so it should time out less often.
2020-08-13 01:50:51 -07:00
a817c9abbe
Redwood metrics was adding details with empty names.
2020-08-12 03:24:23 -07:00
c1ccbc3d27
Bug fixes. Internal page update would be missed if child page ID updates were expected but none were done. When rebuilding internal pages, child page ID updates were being written to the original page memory instead of the rebuild record set.
2020-08-12 01:33:07 -07:00
c18d16ad30
Multiple sibling page updates within the remap cleanup window can now avoid the second update step, copying updated contents over the original page, by instead updating the parent page to point to the new sibling locations.
2020-08-09 03:04:38 -07:00
a49cb41de7
Merge branch 'release-6.3'
...
# Conflicts:
# CMakeLists.txt
# cmake/ConfigureCompiler.cmake
# fdbserver/Knobs.cpp
# fdbserver/StorageCache.actor.cpp
# fdbserver/storageserver.actor.cpp
# flow/ThreadHelper.actor.h
# flow/serialize.h
# tests/CMakeLists.txt
2020-07-29 00:31:55 -07:00
5268dbe1ec
Use knob for remap cleanup window in write speed test.
2020-07-21 00:42:08 -07:00
d05b7ee785
Pager remap remover now accumulates a configurable version interval of page updates behind the oldest retained version which are used to coalesce updates to the same original page ID to reduce write amplification for many workloads.
2020-07-20 04:08:33 -07:00
f470ba8316
Remove using namespace std::rel_ops
...
This causes the following to not compile anymore
\#include <utility>
\#include <vector>
using namespace std::rel_ops;
int main() {
std::vector<int> xs;
return xs.rbegin() != xs.rend();
}
See https://godbolt.org/z/s1977n
2020-07-10 22:58:15 +00:00
1b68665228
Merge branch 'master' into release-6.3
2020-07-08 18:52:05 -07:00
1b55d75896
Remove TRIVIALLY_DESTRUCTIBLE macro
2020-07-04 19:28:10 -07:00
22f7f804b8
Merge branch 'release-6.3' into mengxu/merge-6.3-PR
2020-06-28 11:19:39 -07:00
8de70432a6
Fix memory leaks in VectorRef
...
VectorRef<T> relies on T being trivially destructible, but this was not
yet enforced. By statically asserting that T is trivially destructible,
we avoid leaking memory by not calling the destructor for non-trivially
destructible types
2020-06-24 10:54:19 -07:00
4216112e78
Fixed getWriteVersion and getLastCommittedVersion signatures
2020-06-17 22:38:54 -07:00
77e3b314f6
Mark IKeyValueStore::getStorageBytes const
2020-06-17 18:20:43 -07:00
2a393633b6
Improved const-correctness
...
Using the recently added IndexedSet::const_iterator
(https://github.com/apple/foundationdb/pull/3185 ), we can improve the
const-correctness of many functions. In this PR const is added where
applicable. Also, wherever I came across the following while adding
const, I made the following changes:
- virtual function overrides are marked as override
- NULL is replaced with nullptr
- git clang-format is applied
2020-06-17 18:20:43 -07:00
43d23bccb0
Applied clang-format.
2020-06-16 17:24:32 -07:00
ad68e44419
Change Redwood read concurrency lock to a reference because it must be able to live longer than the storage engine itself due to destruction order of actors if the storage engine is shut down while read operations are in progress.
2020-06-16 03:38:51 -07:00
9888fa7a10
Use a reference for m_commitReadLock to avoid invalid destruction.
2020-06-15 14:46:16 -07:00
c3c7db6a9d
Added flowlock around reads done on the write path so that those reads cannot starve reads to support reads done by the storage server.
2020-06-09 17:00:21 -07:00
8fdb81b48d
Tweaked BTree test random parameter limits to avoid test runs which take too long.
2020-06-06 21:10:52 -07:00
a28b5f0a8b
Possible bug fix, flow locks should be taken after initializing BTree cursor in KVStoreRedwood read functions otherwise it might be possible for the BTree to be closed before the flow lock wait returns, depending on destruction order of some things.
2020-06-06 21:10:13 -07:00
949fbd9145
New low level BTree cursor class which is designed for short lifetimes, does less memory allocation, and can be used to perform getRange() operations with less CPU overhead.
2020-06-04 23:23:14 -07:00
91f3b1c476
Merge pull request #3160 from satherton/feature-redwood
...
Update Redwood.
2020-05-20 15:04:01 -07:00
8c3cddc792
Metrics bug fix, page build count was being updated in an incorrect place.
2020-05-19 12:45:32 -07:00
53c4500694
Applied clang-format.
2020-05-19 04:11:39 -07:00
0d6f226938
Renamed some metrics to be clearer. Instead of "write" and "update" the terms are now "build" and "modify" which are less confusing and better represent what they mean in terms of DeltaTree operations.
2020-05-19 03:31:03 -07:00
7765168346
Redwood metrics now includes average item count, fill percentage, and record stored percentage for pages written or modified during the metrics interval.
2020-05-19 03:04:39 -07:00
FDB Formatster