wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
23,445 Commits 36 Branches 335 Tags 297 MiB
Commit Graph

47 Commits

Author SHA1 Message Date
Yao Xiao f1875268b8
Destroy CF handle before closing physical shard. (#8394) 2022-10-21 17:18:36 -07:00
neethuhaneesha a1eb1d4a48
Rocksdb storage using single_key_deletes instead of deleterange on clearrange operation. (#8452) 2022-10-21 15:47:19 -07:00
He Liu b52edd8658 Merge branch 'main' of https://github.com/apple/foundationdb into validate-data-consistency 2022-10-10 11:00:05 -07:00
He Liu 88c37c81d8 Merge branch 'main' of https://github.com/apple/foundationdb into thread-priority 2022-10-05 14:20:53 -07:00
Markus Pilman 0361cf74e5 fix RocksDB compilation errors 2022-10-05 09:21:15 -06:00
Markus Pilman 550488b020 Merge remote-tracking branch 'origin/main' into bugfixes/open-for-ide 
# Conflicts:
#	bindings/c/CMakeLists.txt
#	fdbclient/include/fdbclient/GetEncryptCipherKeys.actor.h
#	fdbserver/BackupWorker.actor.cpp
#	fdbserver/BlobWorker.actor.cpp
#	fdbserver/CommitProxyServer.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/include/fdbserver/GetEncryptCipherKeys.actor.h
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/PhysicalShardMove.actor.cpp
#	flow/CMakeLists.txt
 2022-10-04 18:27:48 -06:00
neethuhaneesha a565863189
RocksDB compaction knobs and stats. (#8392) 2022-10-04 15:13:50 -07:00
Yao Xiao 957ce81a7d
Add per CF stats. (#8354) 2022-09-30 15:25:29 -07:00
He Liu 5f975623fb Merge branch 'main' of https://github.com/apple/foundationdb into thread-priority 2022-09-30 08:57:53 -07:00
He Liu 63b8d775a3
Knob for RocksDb behaviors (#8360) 
* Added knob for rocksdb suggest compact.

* Added ROCKSDB_LEVEL_COMPACTION_DYNAMIC_LEVEL_BYTES knob.

Co-authored-by: He Liu <heliu@apple.com>
 2022-09-29 15:40:29 -07:00
neethuhaneesha bc9e806f1c RocksDB block cache size increase. 2022-09-21 11:41:27 -07:00
A.J. Beamon 4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
He Liu 2e2c198df7 Added knobs for RocksDB reader/writer thread priorities. 2022-09-19 11:20:17 -07:00
He Liu 958b28497e Merge branch 'main' of https://github.com/apple/foundationdb into validate-data-consistency 2022-09-13 13:55:01 -07:00
Yao Xiao ac7a5823e2
Add knob for CF write buffer size. (#8038) 2022-08-30 17:52:29 -07:00
Steve Atherton 557efa1b53
Merge pull request #7406 from sfc-gh-fzhao/RedwoodDataMovementNonCacheRead 
Redwood add non-cache reads options
 2022-08-30 17:25:05 -07:00
Yao Xiao 09f62acd14
Add delay to physical shard clean up. (#7989) 2022-08-29 11:30:50 -07:00
Fuheng Zhao 7c6dbaf3cf update trState ReadOption to optional 2022-08-24 15:04:17 -07:00
Fuheng Zhao 4e748d6bed use optional ReadOptions and RangeReadOptions in requests 2022-08-23 17:16:47 -07:00
Fuheng Zhao 78f4b4f739 add RangReadOptions which inherit from ReadOptions 2022-08-22 11:46:01 -07:00
He Liu 044f43b6c0
Fix shard mapping read (#7897) 
* Read full range.

* Count empty empty shards.

* Added ROCKSDB_READ_RANGE_ROW_LIMIT.

* Fixed BytesLimit issue.

* TraceEvents.

* Cleaned up comments.

Co-authored-by: He Liu <heliu@apple.com>
 2022-08-19 09:33:59 -07:00
He Liu 7cca369f66 Implemented validateRangeAgainstServer. 2022-08-17 16:51:41 -07:00
Yao Xiao 9c20a07d35
Add delete_obsolete_files_period_micros to RocksDB options. (#7908) 2022-08-17 11:21:21 -07:00
He Liu 19ec3d31e7 Added validateRange() in SS. 2022-08-15 12:59:24 -07:00
Yao Xiao 599e4b86d5 Add more knobs 2022-08-11 15:13:42 -07:00
Yao Xiao fcfe6a8c29 Fix metrics and add knobs 2022-08-11 11:46:21 -07:00
Fuheng Zhao d24a52952c update readOption struct 2022-08-10 10:16:26 -07:00
Fuheng Zhao 1a5b8fbd30 resolve conflicts 2022-08-09 09:57:11 -07:00
Fuheng Zhao e4fb565057 use readOptions to pass type and cacheResult 2022-08-08 15:30:56 -07:00
Yao Xiao 38c79a0eb5
Enable RocksDB metrics (#7739) 
* add metrics

* Pass stats to metric logger

* Add debugID
 2022-08-05 00:38:59 -07:00
He Liu 97cf3c99e7
Remove optimizeForSmallDB. (#7767) 
Co-authored-by: He Liu <heliu@apple.com>
 2022-08-03 13:51:57 -07:00
He Liu 35a4cb91d5
Disable ShardedRocksDB in simulation when shard_encode_location_metadata is disabled (#7726) 
* Disabled tests for ShardedRocks.

Cleaned up ShardedRocks TraceEvent.

Added assertion in ShardManager::validate().

* Added test trace.

* Make sure TraceEvent contains `ShardedRocks`.

* Exclude ShardedRocksDB when SHARD_ENCODE_LOCATION_METADATA is disabled.

Co-authored-by: He Liu <heliu@apple.com>
 2022-07-28 13:54:29 -07:00
He Liu edbc373815
Get ShardedRocks ready for simulation test. (#7679) 
* Disabled tests for ShardedRocks.

Cleaned up ShardedRocks TraceEvent.

Added assertion in ShardManager::validate().

* Added test trace.

* Make sure TraceEvent contains `ShardedRocks`.

Co-authored-by: He Liu <heliu@apple.com>
 2022-07-26 21:49:33 -07:00
He Liu 7a8be255cd
Ss shard management (#7340) 
* Storage server shard management with physical shards.

* Cleanup.

* Resolved comments.

* Added `UnlimintedCommitBytes`.

Co-authored-by: He Liu <heliu@apple.com>
 2022-07-22 09:30:44 -07:00
He Liu 257bab7311
Allow read range on null shard (#7654) 
* Fixed ShardedRocks unit test.

* Removed `UNIT` from noSim/ShardedRocksDBTest.toml.

Co-authored-by: He Liu <heliu@apple.com>
 2022-07-21 13:59:49 -07:00
Yao Xiao e367a5a160
Add KVS physical shard metrics. (#7581) 2022-07-13 01:31:05 -07:00
He Liu aa43956fbd
Sharded RocksDB (#7540) 
* Some fixes.

* Enabled ShardedRocksDB in IKeyValueStore.h

* Added unit test for ShardedRocks for \xff\xff key space read/write.

* Resolved comments.

* Return empty read results if the physical shards are not initialized.

Co-authored-by: He Liu <heliu@apple.com>
 2022-07-08 15:17:38 -07:00
Yao Xiao f98d5ae9e9
[ShardedRocksDB] 5/N Generate and persist metadata for range mapping. (#7408) 2022-07-06 15:57:46 -07:00
Yao Xiao cc93dfd289
Add getStorageBytes. (#7485) 2022-06-28 14:37:26 -07:00
He Liu 61d621a4c4
Fix read iterator pool race (#7455) 2022-06-27 14:51:26 -07:00
Yao Xiao 7da26db342
[ShardedRocksDB] 4/N Support removeRange. (#7345) 2022-06-14 13:52:03 -07:00
Yao Xiao 0bb02f6415
[Sharded RocksDB] 3/N Implement functions for range clear. (#7310) 2022-06-09 10:50:39 -07:00
Yao Xiao 5f1a061e3a
Disable rocksdb metrics. (#7327) 2022-06-06 14:27:41 -07:00
Yao Xiao c75611bf94
[ShardedRocksDB] 2/N Support range read and integrate with read iterator pool. (#7266) 2022-06-02 11:03:49 -07:00
Yao Xiao df3d34f8ac
[ShardedRocksDB] 1/N Implement functions for write and single shard read. (#7184) 2022-05-26 10:59:14 -07:00
Andrew Noyes 665e903999
Fix use of uninitialized memory (#7231) 
We branch on the value of cf, so we need to initialize it.
 2022-05-24 14:58:08 -07:00
Yao Xiao 6bf1e3921f
Add skeleton for sharded rocksdb KVS. (#7181) 2022-05-23 15:29:23 -07:00