wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
26,043 Commits 36 Branches 335 Tags 299 MiB
Commit Graph

523 Commits

Author SHA1 Message Date
Xiaoxi Wang 3605d8c74c populate storage metadata for tss 2023-05-01 18:08:08 -07:00
A.J. Beamon 85f5e206a7
Merge pull request #10047 from sfc-gh-ajbeamon/add-metacluster-version 
Add a metacluster version to the MetaclusterRegistrationEntry and validate it when loading the entry
 2023-05-01 12:32:37 -07:00
A.J. Beamon b258159d3a Change enum capitalization. Improve error reporting if we cannot read metacluster registration when fetching metacluster metrics. Improve timeliness of metacluster metrics updates. 2023-05-01 11:21:42 -07:00
Hui Liu bd8c15634e
Create blob connection per tenant for blob restore (#10070) 2023-05-01 10:26:55 -07:00
A.J. Beamon f1cbc86b94 Add a metacluster version to the MetaclusterRegistrationEntry and validate it when loading the entry from the cluster. 2023-04-27 10:04:57 -07:00
Steve Atherton 7f6d5f296a Merge commit 'e318fc260070ba6ba604930b8f259c9b655938ea' into keybackedrangemap 
# Conflicts:
#	flow/include/flow/error_definitions.h
 2023-04-25 14:21:23 -07:00
Hui Liu 08fffd3bc2
Empty check before calling openContainer on it (#10044) 2023-04-24 17:38:56 -07:00
Steve Atherton 53ee26d758 Changed KeyBackedTypes to an actor file. Added TypedKeySelectors for Map and Set classes and getRange() keySelector methods. Added debug macro for KeyBackedTypes. Rewrote KeyBackedRangeMap using keyselectors on KeyBackedMap. 2023-04-18 22:21:19 -07:00
Nim Wijetunga c780d706d1
Remove EKP Interface from ServerDBInfo (#9909) 
Remove ekp interface from ServerDBInfo
 2023-04-06 11:35:47 -07:00
A.J. Beamon 9c786e6d1e
Merge pull request #9854 from sfc-gh-ajbeamon/metacluster-separate-project 
Metacluster refactoring
 2023-04-04 09:43:41 -07:00
Hui Liu f2a406f609
Add blob manifest and mutation log status to "status json" (#9856) 2023-04-03 18:30:13 -07:00
A.J. Beamon 807646675c Refactor the metacluster project into smaller files, and reorganize the namespaces. Move some metacluster and tenant testing helpers into the metacluster project. 2023-03-30 16:20:09 -07:00
A.J. Beamon e61748c7d5 Move metacluster into its own directory and static library 2023-03-30 16:07:49 -07:00
Xiaoxi Wang e48fd10d8d add perpetual wiggle to .team_tracker field 2023-03-20 09:46:36 -07:00
Steve Atherton 216d0be2cf
Add processID, networkAddress, and locality to layer status JSON for Backup Agents. (#9736) 
* Add processID, networkAddress, and locality to layer status JSON for Backup Agents.

* Backup/dr agent determines network address to report in Layer Status only once, when the status updater loop begins, since it is a blocking call which connects to the cluster.  And lots of code cleanup.
 2023-03-17 18:07:03 -07:00
Hui Liu c43f8b3fdc
Refactor - introduce BlobRestoreController for APIs to manage restore state (#9616) 2023-03-08 07:50:30 -08:00
Hui Liu b2d497a3b2 Report restore phase start timestamp 2023-03-03 18:09:51 -08:00
Yi Wu eac757d186
EaR: cleanup encryption knobs (#9386) 
Changes:
* Cleanup all encryption knobs 
* Update simulated cluster to randomly enable encryption with higher probability
 2023-02-18 13:18:20 -08:00
Hui Liu b475121840
Merge pull request #9267 from sfc-gh-huliu/fixerror 
Fix misc blob restore error messages
 2023-01-30 15:29:42 -08:00
Lukas Joswiak 2b5c0ebe7b Add version epoch to status json 
Adds a new `version_epoch` object to `status json`, which includes the
status of the feature, and the current epoch if it is enabled. If the
version epoch is disabled, the `epoch` field will not be present.

```
{
    "client" : {
        ...
    },
    "cluster" : {
        ...
        "version_epoch" : {
            "enabled" : "true",
            "epoch" : "100000"
        },
        ...
    }
}
```
 2023-01-30 13:21:19 -08:00
Hui Liu 6fbabab6aa Fix misc blob restore error messages 2023-01-30 13:01:27 -08:00
Xiaoxi Wang 8266f52dea
Merge pull request #9012 from sfc-gh-xwang/feature/main/wiggleDelay 
Persist accumulated wiggle delay
 2023-01-04 16:14:09 -08:00
Hui Liu e3bf79cf71 Add correctness test for blob restore 2023-01-04 11:10:34 -08:00
Xiaoxi Wang bbcb3cc018 extract KeyBackedConfig, StorageWiggleData class; solve template resolution problem; solve MV txn and native api conflict by splitting RunTransaction file 2023-01-02 23:34:39 -08:00
Andrew Noyes 91a2010a34 Add .cluster.idempotency_ids to status json 2022-12-14 07:37:44 -08:00
Dennis Zhou 853ed4b131 server/status: fix cgroup reporting to fail silently for v2 
There are a few issues here:
1. The cgroup reporting added looks for top level metrics of cgroup v1.
2. Looking at the whole machine tells us nothing about fdbserver.
3. The reporting itself expects the values to be there rather than
   failing silently if we're not on v1. As cgroup v2 is significantly
   more prominent.

This fixes 3, but 1 and 2 are open and quite questionable. These really
ought to be reported by a 3rd party observer and not fdb itself.
 2022-12-06 11:57:29 -08:00
Hui Liu de8285f11d
Merge pull request #8834 from sfc-gh-huliu/status 
Show blob restore in fdbcli status command
 2022-11-18 16:03:30 -08:00
Hui Liu 2e62822183 Show blob restore in fdbcli status command 2022-11-17 14:22:59 -08:00
Xiaoxi Wang 4727449ef0 Merge branch 'main' of https://github.com/apple/foundationdb into fix/main/restoreStats 2022-11-08 15:35:15 -08:00
Xiaoxi Wang 09f77a785f add PrimaryRegion boolean 2022-11-07 12:58:14 -08:00
Xiaoxi Wang e321044310 make template method accomodate multi-thread transaction 2022-11-07 10:32:20 -08:00
Xiaoxi Wang 03a9dd009a fix compilation errors 2022-11-06 22:46:54 -08:00
Lukas Joswiak cf97541135 Fix exclude status of machines in status json 
Previously, `status json` would report a machine as excluded when any of
its processes were excluded. But a machine should only be reported as
excluded when all of its processes are excluded.

I tested this change by running a small, three process test cluster. I
excluded one machine, and verified `status json` reported the overall
machine as not excluded.
 2022-11-04 09:55:51 -07:00
Xiaoxi Wang 8e6a9730ea move StorageWiggleMetrics out; add workload; try to fix the restore/reset bug (not test) 2022-11-03 23:42:44 -07:00
Jingyu Zhou 928be563a2 Add braces 2022-11-03 13:59:00 -07:00
Jingyu Zhou b02509e356 Fix ASan failure heap use after free 
"incomplete_reasons" may already be destroyed.
 2022-11-03 12:21:09 -07:00
Jingyu Zhou d45f7a088b
Merge pull request #8489 from halfprice/zhewu/debug-shard-size 
Use consistencyScanInfoFetcher function and add a timeout
 2022-10-17 20:28:18 -07:00
Zhe Wu 13a28d026a Return should be optional in consistencyScanInfoFetcher() 2022-10-17 14:15:46 -07:00
Zhe Wu 6e389e4feb Use consistencyScanInfoFetcher function and add a timeout 2022-10-17 11:47:47 -07:00
Hui Liu 049df622f1 add a blob migrator 2022-10-13 13:21:45 -07:00
Jon Fu 2fe1d19e95 Merge branch 'main' of github.com:apple/foundationdb into metacluster-status 2022-10-05 11:48:04 -07:00
Markus Pilman 550488b020 Merge remote-tracking branch 'origin/main' into bugfixes/open-for-ide 
# Conflicts:
#	bindings/c/CMakeLists.txt
#	fdbclient/include/fdbclient/GetEncryptCipherKeys.actor.h
#	fdbserver/BackupWorker.actor.cpp
#	fdbserver/BlobWorker.actor.cpp
#	fdbserver/CommitProxyServer.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/include/fdbserver/GetEncryptCipherKeys.actor.h
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/PhysicalShardMove.actor.cpp
#	flow/CMakeLists.txt
 2022-10-04 18:27:48 -06:00
Markus Pilman 97dfc6823f fixed build with OPEN_FOR_IDE 2022-10-04 17:01:02 -06:00
Jon Fu e26be7a099 Merge branch 'main' of github.com:apple/foundationdb into metacluster-status 2022-09-29 11:37:50 -07:00
Jon Fu 6357ad1750 pass info through cc data to populate in status 2022-09-28 16:18:44 -07:00
A.J. Beamon a7c200b16c Add support for sharing mutation streams between a default backup and DR 2022-09-22 11:36:46 -07:00
Xiaoge Su 970463223c
Merge branch 'main' into main 2022-09-20 16:56:56 -07:00
A.J. Beamon 4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
sfc-gh-ngoyal 1bd97fe628
Recruit new singleton for consistency checker. (#5804) 
* Recruit new singleton for consistency checker.

* Recruit the consistency checker only if enabled.

* Add a yield in monitorConsistencyChecker().

* Minor fixes.

* Consistency check workload enhancements.

* Minor fixes and clarifications.

* clang format

* Clang format.

* Minor fixes, cleanup, debug tracing.

* Misc.

* Move the consistency scan information from dbconfig to a key backed object.

* Move consistency scan config out of db cofig to a state object and feature rename.

* ConsistencyCheck workload refactor.

* devFormat

* Update fdbcli/ConsistencyScanCommand.actor.cpp

* Review Comments.

Co-authored-by: negoyal <neelam.goyal@gmail.com>
Co-authored-by: Ata E Husain Bohra <ata.husain@snowflake.com>
 2022-09-16 09:03:06 -07:00
Xiaoge Su 8130fce97f Update code per comments 
Also sort #include in Platform.actor.cpp
 2022-09-12 11:44:41 -07:00