3b5d23ef88
Remove unnecessary temporary objects when appending to vector of pairs
2021-12-01 18:15:16 -08:00
43b3e05fd5
Merge branch 'master' of github.com:apple/foundationdb into delay-prioritized-eviction
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2021-11-29 16:14:43 -08:00
8216ad1e4e
When a cached page is hit, if it is in the prioritized eviction order don't move it to the normal order.
2021-11-28 21:04:35 -08:00
bed25f9571
Delay prioritized eviction of updated pages until after commit completes.
2021-11-28 21:03:44 -08:00
0f5535fce1
Remove explicit tryEvict(x) as it is unused and it is functionally replaced by prioritizeEviction(x).
2021-11-27 03:26:34 -08:00
12e68da635
Merge pull request #6036 from sfc-gh-satherton/redwood-debug-sim-fixes
...
Test-only bug fixes in Redwood along with debug logging detail improvements.
2021-11-22 11:13:28 -08:00
e43ddb09ac
Merge pull request #6034 from sfc-gh-satherton/remap-window-fix
...
Bug fix: incorrect remap cleanup window size in KeyValueStoreRedwood
2021-11-22 09:12:59 -08:00
d2149631d5
Bug fix, the remap cleanup window was being initialized incorrectly in the Redwood KVS wrapper.
2021-11-19 21:40:09 -08:00
3901d60548
Test-only bug fixes in Redwood along with debug logging detail improvements. Added clearRemapQueue() to Pager to more cleanly and reliably expire all old data and process the remap queue, fixing a bug where with certain configuration parameters and a lot of data the DestructiveSanityCheck would fail because it would not run cleanup long enough. Added more parameters to performance/set unit test.
2021-11-19 01:00:14 -08:00
2613ec7561
Expand use of fmt to get rid of %ld usage
2021-11-17 17:03:32 -08:00
07349869d9
Use fmt to address -Wformat warnings
2021-11-17 14:45:48 -08:00
766a05d33c
Merge remote-tracking branch 'origin/master' into add-format-warning
2021-11-17 12:14:01 -08:00
035e0d6e52
Merge branch 'master' into bit-flipping-workload
2021-11-16 14:42:22 -08:00
21c3c585ca
Make file name parameter more user friendly in unit tests.
2021-11-16 04:03:11 -08:00
867999a41a
Rename wrong_format_version to unsupported_format_version.
2021-11-16 03:25:54 -08:00
7b29804a5e
Fix typo.
2021-11-16 02:30:57 -08:00
c53f5aa110
Renamed redwood to redwood-1-experimental and file extension to .redwood-v1.
2021-11-16 02:15:22 -08:00
508429f30d
Redwood chunked file growth and low priority IO starvation prevention ( #5936 )
...
* Redwood files now growth in large page chunks controlled by a knob to reduce truncate() calls for expansion. PriorityMultiLock has limit on consecutive same-priority lock release. Increased Redwood max priority level to 3 for more separation at higher BTree levels.
* Simulation fix, don't mark certain IO timeout errors as injected unless the simulated process has been set to have an unreliable disk.
* Pager writes now truncate gradually upward, one chunk at a time, in response to writes, which wait on only the necessary truncate operations. Increased buggified chunk size because truncate can be very slow in simulation.
* In simulation, ioTimeoutError() and ioDegradedOrTimeoutError() will wait until at least the target timeout interval past the point when simulation is sped up.
* PriorityMultiLock::toString() prints more info and is now public.
* Added queued time to PriorityMultiLock.
* Bug fix to handle when speedUpSimulation changes later than the configured time.
* Refactored mutation application in leaf nodes to do fewer comparisons and do in place value updates if the new value is the same size as the old value.
* Renamed updatingInPlace to updatingDeltaTree for clarity. Inlined switchToLinearMerge() since it is only used in one place.
* Updated extendToCover to be more clear by passing in the old extension future as a parameter. Fixed initialization warning.
2021-11-12 13:47:07 -08:00
62efeb6812
Merge remote-tracking branch 'origin/master' into add-format-warning
2021-11-12 11:50:36 -08:00
470896bdc4
Redwood inline same-size value updates ( #5925 )
...
* Refactored mutation application in leaf nodes to do fewer comparisons and do in place value updates if the new value is the same size as the old value.
* Renamed updatingInPlace to updatingDeltaTree for clarity. Inlined switchToLinearMerge() since it is only used in one place.
2021-11-10 08:22:57 -08:00
a246c1555d
Merge pull request #5931 from sfc-gh-mpilman/features/all-unit-tests-in-ci
...
Make sure unit tests are run often enough
2021-11-10 08:41:04 -07:00
7df059570a
Make sure unit tests are run often enough
2021-11-08 15:43:32 -07:00
d97d968176
Added KeyBackedObjectMap and KeyBackedObjectProperty classes for storing serializable objects in FDB ( #5896 )
...
* Cleaned up some lambda capture workaround since x=y captures weren't available when these classes were originally written.
* Added KeyBackedObjectMap and KeyBackObjectProperty, which work like KeyBackedMap and KeyBackedProperty but use ObjectWriter/Reader for Value serialization so that the type can evolve over time.
* Disabled unit tests which shouldn't run as part of random selection.
2021-11-08 13:04:53 -08:00
b4d69610ee
Remove unused variable and more clearly explain out of range annotation in Redwood debug output.
2021-11-02 21:48:37 -07:00
84854761cb
Change Redwood to use xxhash for checksums.
2021-11-02 21:47:31 -07:00
ebcc023b6f
Enable missing-field-initializers clang warning
2021-11-01 14:18:31 -07:00
13bb7838aa
Enable clang -Wformat warning
2021-10-30 21:07:38 -07:00
1e7338b6c3
Merge branch 'master' into bit-flipping-workload
2021-10-28 14:24:49 -07:00
0794ebb7e8
Bump format versions for Pager and BTree as the multi-page BTree nodes are now stored in an incompatible way.
2021-10-26 22:17:55 -07:00
b75edbda31
Merge branch 'master' of github.com:apple/foundationdb into RedwoodSuperpage
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2021-10-25 00:55:52 -07:00
9e06b6e6e3
Make IClosable interface const-correct
2021-10-18 13:40:47 -07:00
9713a14ef1
Reverse order of read type and debug ID args
2021-10-18 12:23:09 -04:00
df53cc9580
Add an enum to IKeyValueStore to indicate the source/priority of the read
2021-10-15 14:35:59 -04:00
f913dfed97
Merge branch 'master' into bit-flipping-workload
2021-10-11 16:34:57 -07:00
0cce774325
Merge pull request #5732 from sfc-gh-satherton/kvs-write-version
...
Refactored how Redwood handles commit version
2021-10-09 20:26:52 -07:00
abc14c4921
Redwood crash after shutdown which occurs if a queue page disk read was in progress during pager shutdown but the callback was fired after shutdown completed.
2021-10-09 02:04:23 -07:00
4722412e51
TEST macro logic fix, and moved wait on seek future to the conditional block where it isn't ready.
2021-10-08 08:47:49 -07:00
fd02cbc08a
Bug fix, atomic update case condition for BTree node builds resulting in 1 output node of the same 1-page size as the original was incorrect so atomic updates were never being done.
2021-10-08 00:41:09 -07:00
6981f6f5f2
Merge branch 'master' of github.com:apple/foundationdb into RedwoodSuperpage
2021-10-08 00:28:08 -07:00
c1f05b82ba
Added test coverage with comments for range read seek locking behavior.
2021-10-08 00:06:14 -07:00
8e149f5aa2
Merge branch 'master' of github.com:apple/foundationdb into RedwoodIO
2021-10-07 23:56:45 -07:00
e358a4314d
Added comment about not locking point reads.
2021-10-07 23:55:59 -07:00
45e19c8072
Merge branch 'master' of https://github.com/apple/foundationdb into kvs-write-version
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2021-10-07 20:48:21 -07:00
93f4e01258
Restored Redwood KVS internal auto-incremented version.
2021-10-06 13:15:50 -07:00
b8508e40e9
Added random same-version zero-change commits to unit test.
2021-10-05 02:43:10 -07:00
5be84f1918
Initial Redwood post-create recovery version changed to -1, which means no commit has ever completed. Commits at the same version as the previous commit must have no changes and are no-ops. KeyValueStoreRedwood no longer advances the commit version automatically.
2021-10-05 02:20:42 -07:00
3a56b13989
Bug fix, range end seek in verification map was assuming that version 0 was lower than the minimum mutation version but that is no longer true due to recent changes in commit version behavior.
2021-10-03 22:11:33 -07:00
53ef574343
Many debug changes to Redwood unit test, adding and reformatting output. Errors are no longer counted and accumulated and instead ASSERT(false) is used for any failure so that --crash will end the test immediately.
2021-10-03 22:08:41 -07:00
02227209b3
Added explicit named TraceEvent .log() calls.
2021-09-30 10:06:20 -07:00
d7a420a8d1
Added Redwood pager and BTree overview statistics to trace events logged on creation and recovery. Changed getStorageBytes() to treat internal space used by free list queues as reusable.
2021-09-29 21:01:21 -07:00
sfc-gh-tclinkenbeard