Commit Graph

591 Commits

Author SHA1 Message Date
Lukas Joswiak bfb1c51299 Add `clearknob` fdbcli command
The `clearknob` command clears the value that a knob has been set to in
the configuration database. Note that this does not mean the knob value
itself gets cleared - only the value in the configuration database is
cleared. The value of the knob will revert to whatever is hardcoded in
the corresponding `*Knobs.cpp` file.

Sample `fdbcli` session:

```
Welcome to the fdbcli. For help, type `help'.
fdb> getknob min_trace_severity
`min_trace_severity' is not found
fdb> setknob min_trace_severity 20
Please set a description for the change. Description must be non-empty
description: test
Committed (2)
fdb> getknob min_trace_severity
`min_trace_severity' is `20'
fdb> clearknob min_trace_severity
Please set a description for the change. Description must be non-empty
description: clear
Committed (4)
fdb> getknob min_trace_severity
`min_trace_severity' is not found
```

Transactions are also supported with the new `clearknob` command:

```
Welcome to the fdbcli. For help, type `help'.
fdb> begin
Transaction started
fdb> setknob min_trace_severity 20
fdb> clearknob min_trace_severity
fdb> commit
Please set a description for the change. Description must be non-empty.
description: test
Committed (16)
fdb> getknob min_trace_severity
`min_trace_severity' is not found
```
2023-08-31 17:36:05 -07:00
Jingyu Zhou 22a3ea803c
Add "checkall" debug command for fdbcli (#10687)
* Add "checkall" for checking \xff\x02/blog/ keys

* Avoid GRV calls for getlocation

* Update comments

* Add non-stopping checking and remove verbose output

* Update checkall command to accept customized range

* Fix format

* Fix a compiling issue and output
2023-07-26 17:19:16 -07:00
Jingyu Zhou 614686f737 Add getlocation and getall fdbcli debug commands
getlocation: returns the SS list for a key
getall: returns both the SS list and values on the SS for a key
2023-06-07 14:36:16 -07:00
Zhe Wang f8f8f72c4e
Add audit storage cancellation (#10386)
* list audits

* cancel audits and corresponding tests

* make audit storage dblock aware

* increase audit retry since we are able to cancel

* fix updateAuditState and fdb github ci

* fmt

* fix fdbcli audit_storage and fix CI issue

* fix fdb cli

* address comments

* fmt
2023-06-06 14:29:53 -07:00
He Liu fc8543125c
Added location_metadata fdbcli to query shard locations, assignements… (#10395)
* Added location_metadata fdbcli to query shard locations, assignements, numbers etc.

* Added `listshards` to get some random physical/non-physical shards.

* Resolved comments.
2023-06-06 10:33:48 -07:00
Hui Liu 7ca13d8f9c
support blob restore in fdbrestore (#10248) 2023-05-19 14:45:14 -07:00
Sam Gwydir 6c16875c34
Add networkoption to disable non-TLS connections (#9984)
* Add networkoption to disable non-TLS connections

* add disable plaintext connection to fdbserver

* python doc

* Formatting

* Add tls disable plaintext connection to client api test

* review

* fix negative test

* formatting

* add TLS support to c client config tests

Adds support for TLS in the client and server separately

* add tests for disable_plaintext_connections

Test TLS and Plaintext Clusters and Clients

* Fix documentation

* Rename option to indicate it is client-only

* clearer formatting

* default to allowing plaintext connections

* add SetTLSDisablePlaintextConnection to go bindings
2023-05-13 00:14:11 +02:00
Steve Atherton 69d6e43354 Added explanation of \u support to fdbcli token parsing. Small tweak to rangeconfig hints. Reformatted rangeconfig help to not be intended because the help printer does its own line wrapping which makes it look very messy. 2023-04-25 13:04:38 -07:00
Steve Atherton b70ff34a66 Move custom shard test setup to a separate function. Add JSON utf-8 escaped bytes to fdbcli token parsing. 2023-04-25 10:48:54 -07:00
Steve Atherton a164f8fa9d Add rangeconfig CLI. 2023-04-19 22:19:55 -07:00
Steve Atherton 53ee26d758 Changed KeyBackedTypes to an actor file. Added TypedKeySelectors for Map and Set classes and getRange() keySelector methods. Added debug macro for KeyBackedTypes. Rewrote KeyBackedRangeMap using keyselectors on KeyBackedMap. 2023-04-18 22:21:19 -07:00
Chaoguang Lin 7273723a43
Add the hotrange fdbcli command (#9570)
* Add the hotrange fdbcli command

* Remove the unnecessary state

* Add the doc about the hotrange command
2023-03-06 14:46:52 -08:00
Ankita Kejriwal 27f9c1e06e Merge branch 'main' of github.com:apple/foundationdb into fdbcli 2023-01-27 15:03:27 -08:00
He Liu 00203c8732
Validate Storage part II (#8471)
* Implemented AuditUtils.actor.cpp

Moved AuditUtils to fdbserver/

* Persist AuditStorageState.

* Passed persisted AuditStorageState test.

* Added audit_storage_error to indicate a corruption is caught.

Throw/Send audit_storage_error when there is a data corruption.

Added doAuditStorage() for resuming Audit.

* Load and resume AuditStorage when DD restarts.

* Generate audit id monotonically.

* Fixed minor issue AuditId/Type was not set.

* Adding getLatestAuditStates.

* Improved persisted errors and added AuditStorageCommand.actor.cpp for
fdbcli.

* Added `audit_storage` fdbcli command.

* fmt.

* Fixed null shared_ptr issue.

* Improve audit data.

* Change DDAuditFailed to SevWarn.

* Sev.

* set SERVE_AUDIT_STORAGE_PARALLELISM to 1.

* Moved AuditUtils* to fdbclient/.

* Added getAuditStatus fdbcli command.

* Refactor audit storage fdb cli commands.

* Added auditStorage in sim.

* Cleanup.

* Resolved comments.

* Resolved comments.

* Test disabling audit for sims.

* Cleanup.

Co-authored-by: He Liu <heliu@apple.com>
2023-01-15 21:46:14 -08:00
Ankita Kejriwal f8cfe69488 Fix formatting in fdbcli.actor.cpp 2023-01-06 17:16:29 -08:00
Ankita Kejriwal db520d423a Add fdbcli commands for checking the status of and clearing old idempotency ids 2022-12-22 23:56:55 -08:00
Ankita Kejriwal fe06114bcd Update the help text for `quota` command 2022-12-14 11:01:33 -08:00
FoundationDB CI 86d6106dc1
format source code after switch to clang 15 2022-12-08 17:26:45 +00:00
Daniel Luan 4be05e5a5d Upgrade C++ Standard to 20 2022-12-06 14:19:06 -08:00
Hui Liu 5834517570 Add fdbcli blobrestore to start the full restore 2022-11-11 08:32:23 -08:00
Kevin Hoxha 2dac017ea2 fdbcli: Cancel checkStatus actor when prompting for description in setknob 2022-10-20 21:31:52 -07:00
sfc-gh-tclinkenbeard 9a1876dd11 Update comment for fdbcli quota command 2022-10-14 12:02:14 -07:00
sfc-gh-tclinkenbeard 3827b1ee62 Merge remote-tracking branch 'origin/main' into add-quota-clear-command 2022-10-14 11:55:45 -07:00
Kevin Hoxha ff1b2df8f6 fdbcli: Add options for knob management
- setknob <knob_name> <knob_value> [config_class]
- getknob <knob_name> [config_class]
- Added new option to begin to specify if it's a configuration txn. Syntax is begin [config-txn]
- Added utility function for converting tuples to string
- Added knobmanagment test in fdbcli_tests.py
2022-10-11 15:32:01 -07:00
sfc-gh-tclinkenbeard d67f3ca3b9 Update documentation for quota clear command 2022-10-09 19:43:44 -07:00
Trevor Clinkenbeard ef13985feb
Merge pull request #8320 from sfc-gh-tclinkenbeard/read-write-fungible
Make read and write quotas fungible
2022-09-28 10:59:37 -07:00
Junhyun Shim fd621c0c86 Make TLSConfig check against coordinator addresses one-way
TLS parameters may be loaded by default system paths, which would break
non-TLS cases
2022-09-27 20:57:38 +02:00
sfc-gh-tclinkenbeard 7fc5c196c4 Make read and write quotas fungible 2022-09-25 21:00:11 -07:00
Josh Slocum 28f3cacfd7
Merge pull request #8285 from sfc-gh-jslocum/bg_history_fdbcli
Adding blobkey history fdbcli debug command
2022-09-22 16:56:59 -05:00
Josh Slocum 36df430b14 Merge branch 'main' into bg_history_fdbcli 2022-09-22 15:49:07 -05:00
A.J. Beamon 97a325adab Add an fdbcli command to get tenant group metadata 2022-09-22 13:24:21 -07:00
Josh Slocum 21c627ba97 Adding blobkey history fdbcli debug command 2022-09-22 13:03:53 -05:00
A.J. Beamon 64c8e0e2b1 Convert fdbcli tenant commands to a single command with subcommands. Update the structure of fdbcli tenant tests. 2022-09-21 16:22:58 -07:00
A.J. Beamon 4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
sfc-gh-ngoyal 1bd97fe628
Recruit new singleton for consistency checker. (#5804)
* Recruit new singleton for consistency checker.

* Recruit the consistency checker only if enabled.

* Add a yield in monitorConsistencyChecker().

* Minor fixes.

* Consistency check workload enhancements.

* Minor fixes and clarifications.

* clang format

* Clang format.

* Minor fixes, cleanup, debug tracing.

* Misc.

* Move the consistency scan information from dbconfig to a key backed object.

* Move consistency scan config out of db cofig to a state object and feature rename.

* ConsistencyCheck workload refactor.

* devFormat

* Update fdbcli/ConsistencyScanCommand.actor.cpp

* Review Comments.

Co-authored-by: negoyal <neelam.goyal@gmail.com>
Co-authored-by: Ata E Husain Bohra <ata.husain@snowflake.com>
2022-09-16 09:03:06 -07:00
Dennis Zhou e6222be6c6 fdbcli: use ApiVersion::LATEST_VERSION instead of hardcoding 2022-09-12 17:08:57 -07:00
A.J. Beamon 726d5215a0
Remove API 720 guards for tenants (experimental feature) and the cluster ID special keys (#8108)
* Remove API 720 guards for tenants (experimental feature) and the cluster ID special keys (no need to guard)

* Enable the relaxed special key access in transactions that need to use special key-space APIs introduced in 7.2
2022-09-08 17:22:36 +02:00
Junhyun Shim bcf0861a0a Make fdbcli check for TLSConfig and coordinator address mismatch
At least one of the coordinator addresses in cluster file must contain ":tls" suffix
if fdbcli's resolved TLS client configuration holds any of the TLS elements (key, cert, or CA)

Conversely, if none of the TLS elements are configured,
at least one of coordinator addresses must be without ":tls" suffix
2022-09-01 15:50:11 +02:00
Chaoguang Lin 3fed0456ca
Add an verify option for \xff\xff/worker_interfaces special keys (#7873)
* Add the verify option for \xff\xff/worker_interfaces

* Remove unused code

* update documentations

* update documentations

* solve comments from review

* update some of the comments to be more clear
2022-08-15 14:05:07 -04:00
Vaidas Gasiunas 79571dd2b4
Testing upgrades to a future version of FDB (#7780)
* Enable configuring the next future protocol version as the current protocol version in FDB client, fdbserver, and fdbcli

* Auto format python files used in upgrade tests

* Add a test for upgrading to a future FDB version

* Emphasize that the options for using future protocol version are intended for test purposes only

* Make the global variable for current protocol version visible only locally

* Refactirng to avoid using currentProtocolVersion() in static intialization

* Update go bindings
2022-08-08 17:29:49 +02:00
A.J. Beamon 7f53b340f5 Merge branch 'main' into feature-metacluster
# Conflicts:
#	bindings/python/tests/fdbcli_tests.py
#	fdbcli/TenantCommands.actor.cpp
#	fdbclient/ClientKnobs.cpp
#	fdbclient/include/fdbclient/ClientKnobs.h
#	fdbclient/include/fdbclient/Tenant.h
#	fdbserver/BlobManager.actor.cpp
#	fdbserver/BlobWorker.actor.cpp
#	fdbserver/TenantCache.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/TenantManagementWorkload.actor.cpp
#	flow/include/flow/error_definitions.h
2022-07-28 11:43:21 -07:00
A.J. Beamon 7c6b3fb0b8 Merge branch 'main' into feature-metacluster 2022-07-27 08:55:10 -07:00
Jon Fu 6dbaf46ec8 restructure test workload impl and map 2022-07-26 13:53:04 -07:00
A.J. Beamon f2a8af9155 Better API version handling for fdbcli tenant commands 2022-07-26 09:04:29 -07:00
A.J. Beamon a64693518a Add support for tenant groups 2022-07-26 09:04:29 -07:00
A.J. Beamon 279296c29f Merge branch 'tenant-metadata-change' into feature-metacluster
# Conflicts:
#	fdbclient/SystemData.cpp
#	fdbclient/Tenant.cpp
#	fdbclient/include/fdbclient/SystemData.h
#	fdbclient/include/fdbclient/Tenant.h
#	fdbclient/include/fdbclient/TenantManagement.actor.h
#	fdbserver/TenantCache.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/TenantManagementWorkload.actor.cpp
2022-07-20 09:18:27 -07:00
Markus Pilman 1de37afd52
Make TEST macros C++ only (#7558)
* proof of concept

* use code-probe instead of test

* code probe working on gcc

* code probe implemented

* renamed TestProbe to CodeProbe

* fixed refactoring typo

* support filtered output

* print probes at end of simulation

* fix missed probes print

* fix deduplication

* Fix refactoring issues

* revert bad refactor

* make sure file paths are relative

* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
A.J. Beamon 860d3843cc Merge remote-tracking branch 'origin/feature-tenant-groups' into feature-metacluster 2022-07-16 19:33:26 -07:00
A.J. Beamon c08592368f Add support for tenant groups 2022-07-15 21:18:41 -07:00
A.J. Beamon 0fe6f6ad36 Merge remote-tracking branch 'origin/update-prefix-encoding-in-tenant-metadata-json' into feature-metacluster
# Conflicts:
#	bindings/python/tests/fdbcli_tests.py
#	fdbcli/TenantCommands.actor.cpp
#	fdbclient/include/fdbclient/TenantManagement.actor.h
#	fdbclient/include/fdbclient/TenantSpecialKeys.actor.h
#	fdbserver/workloads/TenantManagementWorkload.actor.cpp
2022-07-14 14:32:29 -07:00