82c92abca2
Change Noisy Sev30s to Sev20s ( #9180 )
...
change noisy sev30 traces to sev20
2023-01-19 14:06:34 -08:00
00203c8732
Validate Storage part II ( #8471 )
...
* Implemented AuditUtils.actor.cpp
Moved AuditUtils to fdbserver/
* Persist AuditStorageState.
* Passed persisted AuditStorageState test.
* Added audit_storage_error to indicate a corruption is caught.
Throw/Send audit_storage_error when there is a data corruption.
Added doAuditStorage() for resuming Audit.
* Load and resume AuditStorage when DD restarts.
* Generate audit id monotonically.
* Fixed minor issue AuditId/Type was not set.
* Adding getLatestAuditStates.
* Improved persisted errors and added AuditStorageCommand.actor.cpp for
fdbcli.
* Added `audit_storage` fdbcli command.
* fmt.
* Fixed null shared_ptr issue.
* Improve audit data.
* Change DDAuditFailed to SevWarn.
* Sev.
* set SERVE_AUDIT_STORAGE_PARALLELISM to 1.
* Moved AuditUtils* to fdbclient/.
* Added getAuditStatus fdbcli command.
* Refactor audit storage fdb cli commands.
* Added auditStorage in sim.
* Cleanup.
* Resolved comments.
* Resolved comments.
* Test disabling audit for sims.
* Cleanup.
Co-authored-by: He Liu <heliu@apple.com>
2023-01-15 21:46:14 -08:00
bbcb3cc018
extract KeyBackedConfig, StorageWiggleData class; solve template resolution problem; solve MV txn and native api conflict by splitting RunTransaction file
2023-01-02 23:34:39 -08:00
11a096567c
Merge branch 'main' of https://github.com/apple/foundationdb into feature/main/wiggleDelay
2022-12-13 12:28:39 -08:00
a6cd33134a
Create per physical shard tracking data stucture. Not populate yet
2022-12-12 13:25:52 -08:00
aae89c863d
DDTeamCollection.getAverageShardBytes
2022-12-07 10:08:22 -05:00
a8b7a33a64
Randomly attempt duplicate snapshots in SnapTestWorkload
2022-12-05 14:28:44 -08:00
6b77bcfc35
fix merge conflict upstream/main
2022-11-16 07:34:42 -08:00
959f77d03d
Merge pull request #8730 from sfc-gh-xwang/fix/main/restoreStats
...
Fix the omit StorageWiggler.restoreStats() bug
2022-11-15 22:37:39 -08:00
907d7af966
solve merge conflict upstream/main
2022-11-15 14:59:31 -08:00
c03f60c618
Update rare code probe annotations
2022-11-15 13:21:25 -08:00
f997e73758
rename variable and solve some light comments
2022-11-14 13:11:27 -08:00
b79268326a
Merge branch 'main' of https://github.com/apple/foundationdb into feature/main/dataApi
2022-11-11 08:29:22 -08:00
3b4e0056a7
Add a knob to enable/disable storage quota enforcement
2022-11-10 19:35:21 -08:00
105648b888
Merge branch 'main' of github.com:apple/foundationdb into commitproxies
2022-11-09 17:38:30 -08:00
4727449ef0
Merge branch 'main' of https://github.com/apple/foundationdb into fix/main/restoreStats
2022-11-08 15:35:15 -08:00
317f0e9efe
finish workload PerpetualWiggleStatsWorkload
2022-11-08 08:14:21 -08:00
c4d9a81d95
fix resetStats() bug
2022-11-07 13:25:57 -08:00
09f77a785f
add PrimaryRegion boolean
2022-11-07 12:58:14 -08:00
e321044310
make template method accomodate multi-thread transaction
2022-11-07 10:32:20 -08:00
bffa838398
Checkpoint restore with shard ( #8667 )
...
* Added SSPhysicalShard.
* Update physicalShards in StorageServer::addShard().
* Handle notAssigned shard.
* fetchKeys() are not stopped during TerminateStorageServer since
physicalShards are not cleared.
* Fixed addingSplitLeft unset shardId.
* Increased the timeout for Rocks reads in simulation.
* Cleanup.
* set SERVE_AUDIT_STORAGE_PARALLELISM to 1.
* Disabled ValidateStorage test.
* Resolved comments.
Co-authored-by: He Liu <heliu@apple.com>
2022-11-07 10:02:35 -08:00
3a02f919b9
Add some comments in DD and fix some nit
2022-11-07 09:11:16 -08:00
03a9dd009a
fix compilation errors
2022-11-06 22:46:54 -08:00
8e6a9730ea
move StorageWiggleMetrics out; add workload; try to fix the restore/reset bug (not test)
2022-11-03 23:42:44 -07:00
27246dd4e6
Merge branch 'main' of github.com:apple/foundationdb into commitproxies
2022-11-03 18:11:56 -07:00
cd7e5fad00
Fetch the list of tenants over storage quota onto commit proxies from DD
2022-11-03 17:53:42 -07:00
55a3db82b5
update the name, comment and discription of write byte sampling; update the calculation of write bandwidth metrics
2022-11-01 14:56:55 -07:00
7bb823edbe
Replace KeyRange with std::vector<KeyRange> in DataMoveMetaData and ( #8591 )
...
* Replace KeyRange with std::vector<KeyRange> in DataMoveMetaData and
CheckpointMetaData.
* Checked if ranges.empty().
* fmt.
* Resolved some comments.
Co-authored-by: He Liu <heliu@apple.com>
2022-10-28 15:22:55 -07:00
438cfea294
Send list of tenants over storage quota to commit proxies
2022-10-26 13:34:12 -07:00
bb0236433c
Merge pull request #8540 from sfc-gh-xwang/feature/main/storageMetrics
...
Make MockStorageServer serve StorageMetrics related request
2022-10-25 17:29:21 -07:00
5a8adca1f7
solve review comments: mark const; add comments; template abbreviation
2022-10-25 10:56:24 -07:00
7647cea4e5
Improve the storage quota monitor code + add a knob for refresh interval
2022-10-21 15:44:13 -07:00
a8b1154e45
Move the storage quota monitor from DataDistributor to TenantCache
2022-10-21 15:17:41 -07:00
fb014a4834
Merge branch 'main' of github.com:apple/foundationdb into monitorusage
2022-10-18 13:14:47 -07:00
5d57429949
modify boolean definition; add comments
2022-10-17 10:12:00 -07:00
83685e6a61
add status verification; add skipDDModeCheck in getInitialDataDistribution for workload test
2022-10-15 22:52:42 -07:00
88c7304e04
Resolved comments.
2022-10-11 10:24:31 -07:00
b52edd8658
Merge branch 'main' of https://github.com/apple/foundationdb into validate-data-consistency
2022-10-10 11:00:05 -07:00
1931e3266f
Resolve comments.
2022-10-10 10:28:19 -07:00
63f5705560
Merge pull request #8414 from sfc-gh-xwang/feature/main/txnProcessor_team
...
Replace self->cx with self->dbContext() method and trivial renaming
2022-10-10 10:09:36 -07:00
ea1325a552
Merge pull request #8319 from sfc-gh-tclinkenbeard/add-rare-code-probe-annotation
...
Add `rare` code probe decoration
2022-10-07 09:39:00 -06:00
2ad4f29539
dbContext() replace self->cx, remove cx member
2022-10-04 16:39:22 -07:00
df9b21169d
change shared_ptr to Reference
2022-09-27 11:22:47 -07:00
985958c260
Add rare code probe decoration
2022-09-25 15:28:32 -07:00
d9cc7ff44e
merge upstream/main
2022-09-23 14:49:46 -07:00
f9e0230b86
DDQueue constructor with ITxnProcessor
2022-09-21 10:56:22 -07:00
97fd5878d9
change DDTeamCollection constructor
2022-09-20 13:00:28 -07:00
08f4c06450
remove unnecessary creation of transactions; call all cx and transaction methods in DDShardTracker through txnProcessor
2022-09-19 14:41:34 -07:00
0e69b19a6f
Added AuditUtils.actor.h
2022-09-19 12:15:49 -07:00
4fd64630e8
Convert literal string ref instances to use _sr suffix
2022-09-19 11:35:58 -07:00
Nim Wijetunga