wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
23,445 Commits 36 Branches 335 Tags 297 MiB
Commit Graph

158 Commits

Author SHA1 Message Date
neethuhaneesha a1eb1d4a48
Rocksdb storage using single_key_deletes instead of deleterange on clearrange operation. (#8452) 2022-10-21 15:47:19 -07:00
neethuhaneesha 3db0c786e5 RocksDB commit latency sample corrections. 2022-10-12 16:19:49 -07:00
neethuhaneesha 4b238e3985
RocksDB disable WAL for experimentation. (#8443) 2022-10-10 12:32:05 -07:00
He Liu 88c37c81d8 Merge branch 'main' of https://github.com/apple/foundationdb into thread-priority 2022-10-05 14:20:53 -07:00
Markus Pilman 0361cf74e5 fix RocksDB compilation errors 2022-10-05 09:21:15 -06:00
Markus Pilman 550488b020 Merge remote-tracking branch 'origin/main' into bugfixes/open-for-ide 
# Conflicts:
#	bindings/c/CMakeLists.txt
#	fdbclient/include/fdbclient/GetEncryptCipherKeys.actor.h
#	fdbserver/BackupWorker.actor.cpp
#	fdbserver/BlobWorker.actor.cpp
#	fdbserver/CommitProxyServer.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/include/fdbserver/GetEncryptCipherKeys.actor.h
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/PhysicalShardMove.actor.cpp
#	flow/CMakeLists.txt
 2022-10-04 18:27:48 -06:00
neethuhaneesha a565863189
RocksDB compaction knobs and stats. (#8392) 2022-10-04 15:13:50 -07:00
A.J. Beamon e1fe28b78b Switch some usages of LiteralStringRef to use the _sr suffix 2022-09-30 16:04:16 -07:00
He Liu 5f975623fb Merge branch 'main' of https://github.com/apple/foundationdb into thread-priority 2022-09-30 08:57:53 -07:00
He Liu 63b8d775a3
Knob for RocksDb behaviors (#8360) 
* Added knob for rocksdb suggest compact.

* Added ROCKSDB_LEVEL_COMPACTION_DYNAMIC_LEVEL_BYTES knob.

Co-authored-by: He Liu <heliu@apple.com>
 2022-09-29 15:40:29 -07:00
A.J. Beamon 4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
He Liu 2e2c198df7 Added knobs for RocksDB reader/writer thread priorities. 2022-09-19 11:20:17 -07:00
Fuheng Zhao 7c6dbaf3cf update trState ReadOption to optional 2022-08-24 15:04:17 -07:00
Fuheng Zhao 4e748d6bed use optional ReadOptions and RangeReadOptions in requests 2022-08-23 17:16:47 -07:00
Fuheng Zhao 78f4b4f739 add RangReadOptions which inherit from ReadOptions 2022-08-22 11:46:01 -07:00
Fuheng Zhao d24a52952c update readOption struct 2022-08-10 10:16:26 -07:00
Fuheng Zhao 1a5b8fbd30 resolve conflicts 2022-08-09 09:57:11 -07:00
Fuheng Zhao e4fb565057 use readOptions to pass type and cacheResult 2022-08-08 15:30:56 -07:00
He Liu a2dc4e20e9
Store rocksdb::DBOptions and rocksdb::ColumnFamilyOptions to (#7766) 
* Store rocksdb::DBOptions and rocksdb::ColumnFamilyOptions to
SharedRocksDBState.

* Make KeyValueStoreRocksDBTest.toml `UNIT`.

* Added TEST_STORAGE_ENGINE_TYPE

* Added readOptions.
 2022-08-08 13:38:02 -07:00
He Liu 118c2ae806
Fixed memory leak for RocksDb and CheckpointReader. (#7665) 
* Fixed memory leak for RocksDb and CheckpointReader.

* Close kvStore at the end of StorageServerCheckpointRestoreTest.
 2022-07-25 12:36:29 -07:00
neethuhaneesha 3cbb51de21 Adding rocksdb compression and data size stats. 2022-07-14 11:28:20 -07:00
neethuhaneesha d80506fd21 Adding logId to trace events. 2022-06-27 10:41:05 -07:00
neethuhaneesha 410ab6352f Traceevent correction. 2022-06-01 15:21:15 -07:00
He Liu 78dc7c5d77 Resolving comments. 2022-05-19 13:27:21 -07:00
He Liu 54c566e57d Drop RocksDB column family before create a new column family from 
import.
 2022-05-19 13:27:21 -07:00
He Liu 05bbe174c3 Implemented checkpoint(), restore() for KeyValueStoreRocksDB, for the 
new format.
 2022-05-19 13:27:21 -07:00
Neethu Haneesha Bingi 93f9ea9a5b Adding rocksdb throttling counters to trace event. 2022-05-05 17:42:26 -04:00
Zhe Wang 6c9ff6ee5e
Add sharded rocksdb type (#6862) 
* add-sharded-rocksdb-type

* address comments

Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
 2022-04-21 22:53:14 -04:00
Neethu Haneesha Bingi 6543bce8ae RocksDb using aggr property metrics for pendingCompactionBytes. 2022-04-14 18:08:42 -07:00
Zhe Wang 2f75a4bd78
Use actor collection for rocksdb histogram actors. (#6805) 
Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
 2022-04-13 14:41:54 -07:00
neethuhaneesha b7096c410f
Merge pull request #6795 from neethuhaneesha/rocksdb-blocksize 
Adding rocksdb block size option.
 2022-04-08 14:20:54 -07:00
Zhe Wang 37054af7e2
Fix RocksDB Metrics (#6803) 
* fix-metrics-in-rocksdb

* remain-GetIntProperty-for-checkRocksdbState

Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
 2022-04-08 17:07:22 -04:00
Zhe Wang 3d325940ad
Fix data race issue when multithreaded RocksDB store using histogram (#6766) 
* using-ThreadReturnPromiseStream-to-make-histogram-thread-safe

* address-comment-and-verify-functionality-in-cluster

* remove-old-histogram-metrics

* fix-comment

Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
 2022-04-07 00:31:36 -04:00
Neethu Haneesha Bingi 0d05669c22 Adding rocksdb block size option. 2022-04-06 19:15:17 -07:00
Zhe Wang 37c7b3ff18 fix-rocksdb-blockcache-recreation-problem 2022-03-29 12:07:41 -04:00
Zhe Wang 383b0cf0e1 add-rocksdb-memory-usage-metrics 2022-03-23 15:59:32 -04:00
Neethu Haneesha Bingi 790ef9ff36 Adding rocksdb compaction readahead size option. 2022-03-22 13:29:35 -07:00
sfc-gh-tclinkenbeard 1aabbb07d2 Add missing copyright headers 2022-03-16 11:25:02 -07:00
He Liu c3a68d661e
Physical Shard Move (#6264) 
Physical Shard Move part I: Checkpoint creation, transfer and restore.
 2022-03-15 13:03:23 -07:00
neethuhaneesha 212deb05e9
Merge pull request #6501 from neethuhaneesha/rocksdb-CompBytesLimiter 
Rocksdb knobs for compaction, storageserver canCommit() waiting if rocksdb overloaded
 2022-03-07 14:59:34 -08:00
Yao Xiao 36edfb8820
Update error type for timeouts. (#6503) 2022-03-07 13:14:20 -08:00
Neethu Haneesha Bingi 8796a763a5 Rocksdb knobs for compaction, storageserver canCommit() waiting if rocksdb overloaded. 2022-03-04 12:41:17 -08:00
Neethu Haneesha Bingi 83e0368eaa RocksDB increasing background threads to speedup compaction. 2022-03-03 15:14:37 -08:00
Zhe Wang f14e08a991 addRocksDBPerfContextMetrics 2022-02-23 22:29:07 -05:00
Aaron Molitor 96dd86ebf8 update RocskDB and Boost 
add Finduring, and include into fdbserver
add BOOST asio/uring settings to fdbserver compile
move portable rocks, liburing up to be configurable at build time.
 2022-02-09 10:48:18 -06:00
Yao Xiao c605226a56
Add error handling in RocksDB KVS. (#6277) 2022-01-25 13:59:25 -08:00
Neethu Haneesha Bingi 162bce7a58 Rocksdb write rate limiter. 2022-01-18 13:23:00 -08:00
Neethu Haneesha Bingi ef4038fe8d Rocksdb read range iterator pool to reuse iterators. 2022-01-18 02:05:21 -08:00
He Liu 1c2c2783ea Implement single deletion in RocksDB. 
If the target deletion range contains a single key, convert it into a
single deletion, to avoid extra cost in RocksDB due to range deletion.
 2022-01-04 09:59:40 -08:00
Neethu Haneesha Bingi 1f30368e71 KeyValueStoreRocksDB histograms to track latencies 2021-12-21 23:09:46 -08:00