Commit Graph

179 Commits

Author SHA1 Message Date
Jingyu Zhou 2d2a2144f4
Update copyright years to 2013-2026 (#12653)
No functional changes.
2026-01-22 10:49:41 -08:00
Syed Paymaan Raza bafb5ca6a3
Apply latest clang-format (#12080) 2025-04-10 20:29:51 -07:00
Syed Paymaan Raza c3e7542cda Update end year in copyright header 2024-08-02 09:40:11 -07:00
Dimitris Apostolou a88114c222
Fix typos 2024-02-07 01:16:00 +02:00
Evan Tschannen ef682d304e fix IKeyValueStore include 2023-06-16 13:28:40 -07:00
Evan Tschannen 3dd86d6c22 move IKeyValueStore.h to the client 2023-05-10 15:41:47 -07:00
Jay Zhuang b7da2ed16c Fix RangeResult.readThrough misuse
Fix `RangeResult.readThrough` misuses:
1. KeyValueStores do not need to set readThrough, as it will not be
   serialized and return. Also setting it to the last key of the result
   is not right, it should at least be the keyAfter of the last key;
2. Fix NativeAPI doesn't set `RangeResult.more` in a few places;
3. Avoid `tryGetRange()` setting `readThrough` when `more` is false,
   which was a workaround for the above item 2;
4. `tryGetRangeFromBlob()` doesn't set `more` but set `readThrough` to
   indicate it is end, which was following the same above workaround I
   think. Fixed that.
5. `getRangeStream()` is going to set `more` to true and then let the
   `readThrough` be it's boundary.

Also added readThrough getter/setter function to validate it's usage.
2023-04-17 21:37:51 -07:00
neethuhaneesha 1d6908d3b4
Changing single key deletions to delete based on number of deletes instead of bytelimit. (#9867) 2023-04-03 13:55:58 -07:00
Steve Atherton 5c795c3abe Rewrite corrupt block number calculations to be more clear. 2023-03-16 13:02:15 -07:00
A.J. Beamon 99f75a9bb1 Fix possible off-by-one in the simulation upper bound check for page corruption 2023-03-16 09:31:11 -07:00
Yi Wu d3bc2afc8e
EaR: storage server uses encryption DB config (#9115)
The PR is updating storage server and Redwood to enable encryption based on the encryption mode in DB config, which was previously controlled by a knob. High level changes are
1. Passing encryption mode in DB config to storage server
    1.1 If it is a new storage server, pass the encryption mode through `InitializeStorageRequest`. the encryption mode is pass to Redwood for initialization
    1.2 If it is an existing storage server, on restart the storage server will send `GetStorageServerRejoinInfoRequest` to commit proxy, and commit proxy will return the current encryption mode, which it get from DB config on its own initialization. Storage server will compare the DB config encryption mode to the local storage encryption mode, and fail if they don't match
2. Adding a new `encryptionMode()` method to `IKeyValueStore`, which return a future of local encryption mode of the KV store instance. A KV store supporting encryption would need to persist its own encryption mode, and return the mode via the API.
3. Redwood accepts encryption mode from its constructor. For a new Redwood instance, caller has to specific the encryption mode, which will be stored in Redwood per-instance file header. For existing instance, caller is supposed to not passing the encryption mode, and let Redwood find it out from its own header.
4. Refactoring in Redwood to accommodate the above changes.
2023-02-06 14:02:31 -08:00
Xiaoge Su 0a60142160 Extract ProcessInfo, MachineInfo, KillType out from ISimulator 2023-01-24 14:48:42 -08:00
sfc-gh-tclinkenbeard 68f14f017c Fix clang 15 compiler warnings 2022-12-08 13:59:37 -08:00
FoundationDB CI 86d6106dc1
format source code after switch to clang 15 2022-12-08 17:26:45 +00:00
Markus Pilman 2e7385891a fix formatting 2022-11-08 15:26:28 -07:00
Markus Pilman aeceeb9383 Don't "uncorrupt" disk blocks 2022-11-01 13:36:16 -06:00
Markus Pilman fd00a56003 don't corrupt sqlite wal files 2022-10-28 10:35:06 -06:00
Markus Pilman e7b5b870a3 Merge remote-tracking branch 'origin/main' into bugfixes/machines-attrition-debugging 2022-10-24 15:24:36 -06:00
Markus Pilman 43cafb0bc2 Track disk corruptions and mark resulting failures as injected 2022-10-24 14:54:43 -06:00
neethuhaneesha a1eb1d4a48
Rocksdb storage using single_key_deletes instead of deleterange on clearrange operation. (#8452) 2022-10-21 15:47:19 -07:00
Markus Pilman d1c80659b5 Remember disk corruptions and downgrade trace severity if a corruption was injected 2022-10-19 16:18:00 -06:00
Steve Atherton 9a934cf98b Sim-only bug fix where SQLite can take too long to shut down so its files remain open when the next virtual process starts. 2022-10-04 19:29:38 -07:00
A.J. Beamon 4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
Steve Atherton 0e0a10ab45 Merge commit '59b04d46cff720df5d267daa8dc8a60c25466f74' into kaio-latency-pctile 2022-09-15 20:27:02 -07:00
Steve Atherton 2bf90ca5ec Change KAIO latency metrics to use LatencySample for easier usability. Rename a SQLite-specific knob to indicate it is specific to SQLite. 2022-09-15 13:27:23 -07:00
sfc-gh-tclinkenbeard 82adc1e856 Make g_simulator a pointer 2022-09-15 09:00:33 -07:00
Fuheng Zhao 7c6dbaf3cf update trState ReadOption to optional 2022-08-24 15:04:17 -07:00
Fuheng Zhao 4e748d6bed use optional ReadOptions and RangeReadOptions in requests 2022-08-23 17:16:47 -07:00
Fuheng Zhao 78f4b4f739 add RangReadOptions which inherit from ReadOptions 2022-08-22 11:46:01 -07:00
Fuheng Zhao 28469196b5 add comments 2022-08-10 15:45:15 -07:00
Fuheng Zhao 88284b9cd5
Update fdbserver/KeyValueStoreSQLite.actor.cpp ReadOption
Co-authored-by: Nim Wijetunga <nim.wijetunga@snowflake.com>
2022-08-10 14:20:59 -07:00
Fuheng Zhao d24a52952c update readOption struct 2022-08-10 10:16:26 -07:00
Fuheng Zhao 1a5b8fbd30 resolve conflicts 2022-08-09 09:57:11 -07:00
Fuheng Zhao e4fb565057 use readOptions to pass type and cacheResult 2022-08-08 15:30:56 -07:00
Markus Pilman 1de37afd52
Make TEST macros C++ only (#7558)
* proof of concept

* use code-probe instead of test

* code probe working on gcc

* code probe implemented

* renamed TestProbe to CodeProbe

* fixed refactoring typo

* support filtered output

* print probes at end of simulation

* fix missed probes print

* fix deduplication

* Fix refactoring issues

* revert bad refactor

* make sure file paths are relative

* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
Markus Pilman 03d913a1de Flow compiling 2022-06-27 17:05:55 -06:00
Markus Pilman a47ed89018 Linux fixes and addressed review comments 2022-06-23 20:52:13 -06:00
Markus Pilman de48e90276 fdbserver is now compiling 2022-06-23 18:45:26 -06:00
Markus Pilman d35445a868 enforce include modularization in cmake 2022-06-23 14:37:35 -06:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Steve Atherton 8f8f95931b In the SQLite storage engine, destroy and create new cursors after SQLITE_CURSOR_MAX_LIFETIME_BYTES of KV read usage because cursor usage increases page cache memory usage in SQLite (internal page cache, not AsyncFileCached) by pinning pages which are not freed until the cursor is destroyed. 2022-03-07 11:20:59 -08:00
Jingyu Zhou 1a5bf25b5c Update code base to use fmt 8.1.1 2022-03-04 15:52:06 -08:00
Renxuan Wang 233c918ffb Replace printf() and fprintf() with fmt::print(). 2022-02-25 19:06:57 -08:00
Renxuan Wang f7eb66441d Try eliminating warnings in macOS and Windows CI builds.
MacOS warnings are format warnings, e.g., `format specifies type 'long' but the argument has type 'Version' (aka 'long long')`.
Windows warnings are `ACTOR does not contain a wait() statement`.
2022-02-25 19:06:57 -08:00
A.J. Beamon 250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
Ray Jenkins dd45805312
Merge branch 'apple:main' into threadname-issue-6064 2022-02-01 17:40:07 -06:00
gaozengqi f1bd0f16da Run clang-format 2022-01-28 10:40:33 -08:00
gaozengqi 0e6dbf434b Fix use allKeys.end 2022-01-28 10:40:33 -08:00
gaozengqi 1419d1aab7 Fix short hand 2022-01-28 10:40:33 -08:00
gaozengqi 697d075e20 Add role 'kvfiledump' to dump key-values from storage file 2022-01-28 10:40:33 -08:00