8f844437da
Avoid unused variable warning when debug output is not on.
2022-03-07 23:12:32 -08:00
9f690d5bd5
Added StringRef::same() which checks data pointers and lengths for match. Fixed a false negative (but not a bug) in boundariesNormal() and used same() to avoid string comparisons.
2022-03-07 22:44:24 -08:00
35df00eba2
Re-enable simulation-only boundary verification.
2022-03-07 22:31:29 -08:00
55c98a4287
Large refactor of redwood debug output to improve context and readability.
2022-03-07 22:30:20 -08:00
11f25bc08f
Initialize decode boundary verifier to nullptr for restart tests, also disable check for now as the feature is not finished.
2022-03-04 17:20:31 -08:00
9f1d4580e4
Added simulation-only verification of DeltaTree initialization boundaries between writes and reads over time.
2022-03-01 23:46:38 -08:00
ee5030d06c
Fixed memory "leak" where child node update tracker entries for freed internal pages were not removed so the map would just grow.
2022-02-28 00:16:10 -08:00
398374c7d7
Rename cacheSizeBytes to pageCacheBytes to match other tests.
2022-02-27 00:36:23 -08:00
73a81a86da
Added Redwood metrics printing during set unit test using new generic repeatEvery() actor.
2022-02-26 23:37:30 -08:00
3a666610ad
Tweaked Redwood unit test ops limits and added total record read count to prevent test instances that run too long.
2022-02-26 23:36:37 -08:00
6f16702042
Merge branch 'main' of https://github.com/apple/foundationdb into redwood-bug-fixes-and-memory-leak
2022-02-26 18:00:18 -08:00
c122cf1ce0
Bug fix: Avoid having a local temporary Cursor destructed after the Reference<Page> containing its DecodeCache has been dropped by using the BTreeCursor's leaf cursor by reference.
2022-02-26 02:03:43 -08:00
a47f481bae
Bug fix, the extent cache used during recovery is not allowed to evict anything so it now uses a private Evictor with no size limit.
2022-02-25 19:21:11 -08:00
276cf3a402
Fix minor bugs turned up by _GLIBCXX_DEBUG ( #6375 )
...
* Fix a benign bug turned up by _GLIBCXX_DEBUG
Just calling std::vector::operator[] with an out of bounds index at all
is technically undefined behavior.
* Fix compilation issue with _GLIBCXX_DEBUG
For some reason std::max with an initializer list isn't constexpr with
_GLIBCXX_DEBUG set
2022-02-25 13:50:41 -08:00
a32c3a2891
Redwood now uses one global page cache eviction order and size limit for all page sizes, for real or virtual processes, and DecodeCache memory is counted against the global page cache budget.
2022-02-25 03:20:53 -08:00
250a88e682
Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement.
2022-02-24 12:25:52 -08:00
a45c9ccce1
Revert "Added Redwood knob to reserve part of configured page cache size for Redwood page decode cache state."
...
This reverts commit 072bc86bb1
2022-02-18 11:14:40 -08:00
17fe322bc4
Added near real-time tracking of Redwood page DecodeCache memory.
2022-02-18 03:26:05 -08:00
072bc86bb1
Added Redwood knob to reserve part of configured page cache size for Redwood page decode cache state.
2022-02-17 16:15:01 -08:00
42136bf808
Fixed memory leak (and minor performance bug) where RedwoodRecordRef::updateCache() would create the key copy to be placed in the cache but not actually place it into the real cache entry because the cache parameter was accepted by value.
2022-02-14 22:47:30 -08:00
3b5d23ef88
Remove unnecessary temporary objects when appending to vector of pairs
2021-12-01 18:15:16 -08:00
43b3e05fd5
Merge branch 'master' of github.com:apple/foundationdb into delay-prioritized-eviction
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2021-11-29 16:14:43 -08:00
8216ad1e4e
When a cached page is hit, if it is in the prioritized eviction order don't move it to the normal order.
2021-11-28 21:04:35 -08:00
bed25f9571
Delay prioritized eviction of updated pages until after commit completes.
2021-11-28 21:03:44 -08:00
0f5535fce1
Remove explicit tryEvict(x) as it is unused and it is functionally replaced by prioritizeEviction(x).
2021-11-27 03:26:34 -08:00
12e68da635
Merge pull request #6036 from sfc-gh-satherton/redwood-debug-sim-fixes
...
Test-only bug fixes in Redwood along with debug logging detail improvements.
2021-11-22 11:13:28 -08:00
e43ddb09ac
Merge pull request #6034 from sfc-gh-satherton/remap-window-fix
...
Bug fix: incorrect remap cleanup window size in KeyValueStoreRedwood
2021-11-22 09:12:59 -08:00
d2149631d5
Bug fix, the remap cleanup window was being initialized incorrectly in the Redwood KVS wrapper.
2021-11-19 21:40:09 -08:00
3901d60548
Test-only bug fixes in Redwood along with debug logging detail improvements. Added clearRemapQueue() to Pager to more cleanly and reliably expire all old data and process the remap queue, fixing a bug where with certain configuration parameters and a lot of data the DestructiveSanityCheck would fail because it would not run cleanup long enough. Added more parameters to performance/set unit test.
2021-11-19 01:00:14 -08:00
2613ec7561
Expand use of fmt to get rid of %ld usage
2021-11-17 17:03:32 -08:00
07349869d9
Use fmt to address -Wformat warnings
2021-11-17 14:45:48 -08:00
766a05d33c
Merge remote-tracking branch 'origin/master' into add-format-warning
2021-11-17 12:14:01 -08:00
035e0d6e52
Merge branch 'master' into bit-flipping-workload
2021-11-16 14:42:22 -08:00
21c3c585ca
Make file name parameter more user friendly in unit tests.
2021-11-16 04:03:11 -08:00
867999a41a
Rename wrong_format_version to unsupported_format_version.
2021-11-16 03:25:54 -08:00
7b29804a5e
Fix typo.
2021-11-16 02:30:57 -08:00
c53f5aa110
Renamed redwood to redwood-1-experimental and file extension to .redwood-v1.
2021-11-16 02:15:22 -08:00
508429f30d
Redwood chunked file growth and low priority IO starvation prevention ( #5936 )
...
* Redwood files now growth in large page chunks controlled by a knob to reduce truncate() calls for expansion. PriorityMultiLock has limit on consecutive same-priority lock release. Increased Redwood max priority level to 3 for more separation at higher BTree levels.
* Simulation fix, don't mark certain IO timeout errors as injected unless the simulated process has been set to have an unreliable disk.
* Pager writes now truncate gradually upward, one chunk at a time, in response to writes, which wait on only the necessary truncate operations. Increased buggified chunk size because truncate can be very slow in simulation.
* In simulation, ioTimeoutError() and ioDegradedOrTimeoutError() will wait until at least the target timeout interval past the point when simulation is sped up.
* PriorityMultiLock::toString() prints more info and is now public.
* Added queued time to PriorityMultiLock.
* Bug fix to handle when speedUpSimulation changes later than the configured time.
* Refactored mutation application in leaf nodes to do fewer comparisons and do in place value updates if the new value is the same size as the old value.
* Renamed updatingInPlace to updatingDeltaTree for clarity. Inlined switchToLinearMerge() since it is only used in one place.
* Updated extendToCover to be more clear by passing in the old extension future as a parameter. Fixed initialization warning.
2021-11-12 13:47:07 -08:00
62efeb6812
Merge remote-tracking branch 'origin/master' into add-format-warning
2021-11-12 11:50:36 -08:00
470896bdc4
Redwood inline same-size value updates ( #5925 )
...
* Refactored mutation application in leaf nodes to do fewer comparisons and do in place value updates if the new value is the same size as the old value.
* Renamed updatingInPlace to updatingDeltaTree for clarity. Inlined switchToLinearMerge() since it is only used in one place.
2021-11-10 08:22:57 -08:00
a246c1555d
Merge pull request #5931 from sfc-gh-mpilman/features/all-unit-tests-in-ci
...
Make sure unit tests are run often enough
2021-11-10 08:41:04 -07:00
7df059570a
Make sure unit tests are run often enough
2021-11-08 15:43:32 -07:00
d97d968176
Added KeyBackedObjectMap and KeyBackedObjectProperty classes for storing serializable objects in FDB ( #5896 )
...
* Cleaned up some lambda capture workaround since x=y captures weren't available when these classes were originally written.
* Added KeyBackedObjectMap and KeyBackObjectProperty, which work like KeyBackedMap and KeyBackedProperty but use ObjectWriter/Reader for Value serialization so that the type can evolve over time.
* Disabled unit tests which shouldn't run as part of random selection.
2021-11-08 13:04:53 -08:00
b4d69610ee
Remove unused variable and more clearly explain out of range annotation in Redwood debug output.
2021-11-02 21:48:37 -07:00
84854761cb
Change Redwood to use xxhash for checksums.
2021-11-02 21:47:31 -07:00
ebcc023b6f
Enable missing-field-initializers clang warning
2021-11-01 14:18:31 -07:00
13bb7838aa
Enable clang -Wformat warning
2021-10-30 21:07:38 -07:00
1e7338b6c3
Merge branch 'master' into bit-flipping-workload
2021-10-28 14:24:49 -07:00
0794ebb7e8
Bump format versions for Pager and BTree as the multi-page BTree nodes are now stored in an incompatible way.
2021-10-26 22:17:55 -07:00
b75edbda31
Merge branch 'master' of github.com:apple/foundationdb into RedwoodSuperpage
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2021-10-25 00:55:52 -07:00
Steve Atherton