2a8d39d5e5
Update ser-des unit test for the SSI boolean
2022-03-21 10:41:28 -07:00
37e7c80f26
Merge branch 'main' into blob_integration
2022-03-17 18:45:42 -05:00
c3a68d661e
Physical Shard Move ( #6264 )
...
Physical Shard Move part I: Checkpoint creation, transfer and restore.
2022-03-15 13:03:23 -07:00
c635dcd3ad
Add tenant support in the FDB native client
2022-03-15 09:21:27 -07:00
8c35ceb684
switching bg file keys to sort on version then type, and using that to fix race where 2 blob workers try to split the same granule at different versions
2022-03-14 17:38:31 -05:00
72d99cc5b1
Create new system key for tracking latest software version installed on the cluster
2022-03-13 20:03:33 -07:00
e71b3533f9
Merge branch 'main' into blob_integration
2022-03-09 08:59:56 -06:00
5fa9d3e1b7
Add a tenant parameter to read and commit requests. Store a map of all tenants on commit proxy and storage servers. Add an option to require tenant mode.
2022-03-06 21:54:21 -08:00
8f8987ad67
Merge branch 'blob_integration' of https://github.com/apple/foundationdb into blob_integration
2022-03-01 10:12:36 -08:00
ed9f95b299
simplifying blob manager recovery by always treating the DB granule mapping as the source of truth, and persisting all boundary changes there BEFORE updating workers
2022-02-25 14:46:14 -06:00
330b2b48ec
improved file cleanup execution and testing
2022-02-22 12:00:09 -08:00
38a75a8b89
Merge branch 'main' into blob_integration
2022-02-17 17:47:38 -06:00
092b5cee4b
MVC2.0: Rollback added code
2022-02-14 13:50:42 -08:00
6dc5921575
createdTime based storage wiggler ( #6219 )
...
* add storagemetadata
* add StorageWiggler;
* fix serverMetadataKey bug
* add metadata tracker in storage tracker
* finish StorageWiggler
* update next storage ID
* change pid to server id
* write metadata when seed SS
* add status json fields
* remove pid based ppw iteration
* fix time expression
* fix tss metadata nonexistence; fix transaction retry when retrieving metadata
* fix checkMetadata bug when store type is wrong
* fix remove storage status json
* format code
* refactor updateNextWigglingStoragePID
* seperate storage metadata tracker and store type tracker
* rename pid
* wiggler stats
* fix completion between waitServerListChange and storageRecruiter
* solve review comments
* rename system key
* fix database lock timeout by adding lock_aware
* format code
* status json
* resolve code format/naming comments
* delete expireNow; change PerpetualStorageWiggleID's value to KeyBackedObjectMap<UID, StorageWiggleValue>
* fix omit start rount
* format code
* status json reset
* solve status json format
* improve status json latency; replace binarywriter/reader to objectwriter/reader; refactor storagewigglerstats transactions
* status timestamp
2022-02-04 15:04:30 -08:00
13da5d394b
Remove extra StringRef copy
2022-01-26 19:22:40 -08:00
e2e853a30b
Use ObjectWriter to write storage cache data
...
This allows deserialization of the on disk data in lower FDB versions
that don't contain fields added in main.
2022-01-25 19:24:14 -08:00
1494f8216b
Reworked blob manager recovery to be more efficient and handle overlapping ongoing splits
2022-01-24 14:12:45 -06:00
08f8700636
Fixed a couple bugs in blob manager recovery
2022-01-18 18:47:42 -06:00
9d9cb961a1
reworked blob manager recovery to be more efficient
2022-01-18 14:22:58 -06:00
932f68e1b3
Refactor bstore and add watch key.
2021-12-10 16:43:58 -06:00
22d72ec9dd
init
2021-12-10 16:43:58 -06:00
efb21ca6a1
Merge branch 'master' into blob_integration
2021-11-24 13:17:54 -06:00
964d0209ca
Merge pull request #5637 from sfc-gh-ljoswiak/features/data-loss-prevention
...
Data loss protection when joining new cluster
2021-11-15 15:26:32 -08:00
f140028086
Merge branch 'master' into blob_integration
2021-11-15 15:09:40 -06:00
30867750b5
Add protection against storage and tlog data deletion when joining a new cluster
2021-11-09 12:29:47 -08:00
3b711af061
Fixed a couple rollback issues and endianness of versionstamp version
2021-10-29 12:07:16 -05:00
875824b186
MVC2.0: Notify clients about relevant changes of client libraries
2021-10-27 23:43:40 +02:00
40da5a80f9
Merge remote-tracking branch 'apple/master' into multi-version-client-2
2021-10-26 19:29:10 +02:00
37bc41abbb
Merge remote-tracking branch 'apple/master' into multi-version-client-2
2021-10-26 18:51:43 +02:00
773886515e
Merge branch 'feature-range-feed' into blob_full_clean
2021-10-22 11:07:51 -05:00
3f7df58a77
fixed a number of issues
2021-10-19 13:56:52 -07:00
912ef76f1c
cleanup before merge
2021-10-18 17:11:14 -05:00
b5074fd597
Reworked all of the system data to encode granule data more efficiently for persistence
2021-10-13 16:28:04 -05:00
5f0ec0612a
Merge branch 'feature-range-feed' into blob_full
2021-10-13 15:44:35 -05:00
5b6f8b2abb
added the known committed version to change feeds
2021-10-11 13:53:36 -07:00
5c642f706e
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
2021-10-09 19:34:16 -07:00
cda0a5f931
Operation to upload client library binary in to system keyspace
2021-10-06 18:01:46 +02:00
3e79299898
Locality filter support to perpetual storage wiggler feature.
2021-09-30 10:00:33 -07:00
a986679b7f
More bug fixes
2021-09-24 10:02:02 -05:00
5ddf08dfe5
Got basic range reassignment working
2021-09-22 16:48:44 -05:00
a1f6dcc2d5
merge from master
2021-09-22 13:07:38 -07:00
914b1ee960
fmt
2021-09-20 12:01:47 -07:00
95f64f97de
resolved some fmt issues
2021-09-20 11:58:30 -07:00
abf73047ca
Enforce std:: specifier rather than using namespace
2021-09-16 19:40:28 -07:00
cf1626f553
Added comments.
2021-09-10 11:10:10 -07:00
c2d1d1704f
Merge branch 'feature-range-feed' into blob_full
2021-09-10 11:21:52 -05:00
2ea055371b
Introduced serverKeysTrueEmptyRange to indicate the SS that the new
...
shard is empty.
2021-09-09 14:15:07 -07:00
eb76343dfb
Added blob granule reassignment and splitting
2021-09-08 14:09:14 -05:00
4bbae59bb0
Fixed a few bugs and added a backup popping mechanism
2021-09-02 21:11:44 -07:00
46adada5ff
Cleaned up debugging and fixed a couple bugs
2021-08-31 12:30:43 -05:00
b4bfd58bcb
multiple blob workers appears to work
2021-08-30 13:07:25 -05:00
3b011408f8
Added sequence numbers and locks to blob worker and manager
2021-08-27 16:33:07 -05:00
a278d2977a
renamed range feeds to change feeds, data distribution support almost complete
2021-08-13 14:27:15 -07:00
76a3c47e42
Got full demo working
2021-08-12 10:18:58 -05:00
df6bbb6a08
fix: needed bigEndian instead of littleEndian
2021-08-10 14:37:05 -07:00
921a2cfca1
Merge branch 'feature-range-feed' into blob_full
2021-08-10 11:25:48 -05:00
f1eedf4910
fixed serialization of rangefeed durable keys
2021-08-09 17:15:51 -07:00
e5b0cf20aa
Reader can read blob worker interface and mapping from db, read files, apply mutations, and return result
2021-08-06 11:02:05 -05:00
e83b27387a
POC worker that dumps fdb range to an s3 object
2021-08-06 11:01:37 -05:00
a931f70cfb
Added fdbcli for blobrange
2021-08-06 11:01:37 -05:00
0989c28a6b
made range feeds durable on the storage server
2021-07-30 15:23:42 -07:00
406562b282
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
2021-07-27 17:04:58 -07:00
507c1f11e3
Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use.
2021-07-26 19:55:10 -07:00
15fe00927a
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/StorageServerInterface.h
# fdbclient/SystemData.cpp
# fdbclient/SystemData.h
# fdbserver/storageserver.actor.cpp
# flow/ProtocolVersion.h
2021-07-12 19:10:14 -07:00
ab0ab765c8
Merge pull request #4965 from sfc-gh-jslocum/tss_mapping_changes
...
TSS Mismatch Changes
2021-06-29 08:24:25 -07:00
4ad5926a25
Snake naming of keys and added comments to all new functions.
2021-06-23 18:03:27 -07:00
73752f441b
exclude locality:clang-format, ranged loops, documentation, tracking addStoragesever for exclusion.
2021-06-23 18:03:27 -07:00
62355571d0
exclude servers based on locality match
2021-06-23 18:03:27 -07:00
826e70916c
Merge remote-tracking branch 'origin/master' into config-db
2021-06-17 09:47:41 -07:00
56dadaa428
TSS Mismatch Changes
2021-06-11 23:13:16 +00:00
65c49fc1e4
Merge pull request #4934 from sfc-gh-jslocum/tss_quarantine
...
Added TSS Quarantine and fixed TSS mapping bug in CommitProxy
2021-06-11 11:52:42 -07:00
41c790b299
Merge remote-tracking branch 'origin/master' into config-db
2021-06-10 22:31:23 -07:00
c6d96c4124
Added TSS Quarantine and fixed TSS mapping bug in CommitProxy
2021-06-09 14:42:50 +00:00
866f536983
Merge pull request #4888 from sfc-gh-tclinkenbeard/remove-fdbserver-includes
...
Remove fdbserver includes from fdbclient
2021-06-07 10:22:13 -07:00
b3e4f182ef
TSS Mapping Change
2021-06-02 17:30:09 +00:00
a775f92fca
Merge remote-tracking branch 'origin/master' into config-db
2021-06-01 15:39:34 -07:00
1a40c60674
Move RestoreWorkerInterface into fdbserver
2021-05-30 15:02:33 -07:00
594e8944ae
Move RestoreWorkerInterface into fdbserver
2021-05-30 11:51:47 -07:00
9515e8d44f
Add special configuration database keys to SystemData.h
2021-05-29 11:48:47 -07:00
4257ac2b4d
More TSS Changes/Fixes
2021-05-25 20:37:48 +00:00
ce82c9653e
Testing Storage Server implementation
2021-05-25 20:28:50 +00:00
6165af81fa
Merge branch 'apple:master' into feature-range-feed
2021-05-21 15:21:34 -07:00
a57061a5ed
change UID to PID meaning Process ID
2021-05-20 00:06:53 +00:00
6065d247f8
fix perpetualStorageWiggleKey
2021-05-17 20:59:38 +00:00
5c2d7b6080
Create RangeResult type alias
2021-05-03 13:14:16 -07:00
78697003ea
more formatting
2021-04-30 10:51:35 -07:00
d652287bae
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbclient/DatabaseContext.h
# fdbclient/NativeAPI.actor.cpp
# fdbclient/StorageServerInterface.h
# fdbserver/storageserver.actor.cpp
2021-04-30 10:29:42 -07:00
6de28dd916
clang-format
2021-04-14 10:56:33 -07:00
c3f68831af
Move existing ClientDBInfo variables to global configuration
2021-04-14 10:56:32 -07:00
80c6048a01
Naming fixes
2021-04-14 10:56:32 -07:00
f1415412f1
Add global configuration framework implementation
2021-04-14 10:56:32 -07:00
df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
2bb4f2e59f
Merge branch 'release-6.3-pre-format' into master-format
...
This merges release-6.3 branch right before it was fully formatted.
There were quite a few conflicts that are resolved here. CoroFlow had
a check for OOM errors introduced in 6.3, but didn't seem applicable in
the new implmentation which seems to use boost.
2021-03-10 09:37:41 -08:00
0c72e5a9cb
fixed name of function
2021-03-04 20:22:42 -08:00
bc757f3e4b
basic framework for range feed support
2021-03-04 19:14:54 -08:00
37d9e975e9
Fix multiple compiler warnings
2021-03-03 10:18:03 -07:00
ff7d306b09
Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
...
Include conflict markers for now. Will resolve.
2021-01-15 18:04:09 +00:00
53d23d95d8
initial commit in adding more documentation to special keys
2021-01-04 15:26:48 -05:00
bbb56e4089
Merge branch 'release-6.2' of https://github.com/apple/foundationdb into release-6.3
2020-12-23 14:26:59 -08:00
ac02329d7d
Added a command in fdbcli to allow user to manually trigger the detailed teams info loggings in data distributor
2020-12-03 14:42:30 -08:00
Bharadwaj V.R