wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
foundationdb/fdbclient
History
Michael Stack ff22876247
Add multiparting to s3client. (#11920) 
* Add multiparting to s3client.
Fix boost::urls::parse_uri 's dislike of credentialed blobstore urls.

* fdbclient/BulkLoading.cpp
 Add blobstore regex to extract credentials before feeding the boost
 parse_uri.

* fdbclient/include/fdbclient/S3BlobStore.h
* fdbclient/S3BlobStore.actor.cpp
 Add cleanup of failed multipart -- abortMultiPartUpload l(s3 will do
 this in the background eventually but lets clean up after ourselves).
 Also add  getObjectRangeMD5 so can do multipart checksumming.

* fdbclient/S3Client.actor.cpp
 Change upload file and download file to do multipart always.
 Retry too.

* fdbclient/S3Client_cli.actor.cpp
 Add command line to trace rather than output.

* Address Zhe review

* More logging around part upload and download

* Undo assert that proved incorrect; restore the old length math
doing copy in readObject.

Cleanup around TraceEvents in HTTTP.actor.

* Undo commented out cleanup -- for debugging

* formatting

---------

Co-authored-by: stack <stack@duboce.com>
 		2025-02-13 09:06:17 -08:00
..
azure_backup Update end year in copyright header 2024-08-02 09:40:11 -07:00
include/fdbclient Add multiparting to s3client. (#11920) 2025-02-13 09:06:17 -08:00
sha1 fixed link errors 2022-06-27 19:00:40 -06:00
tests Bulkload Engine Support General Storage Engine and Fix BulkLoad Bugs (#11898) 2025-02-06 12:04:13 -08:00
vexillographer More copyright end year updates (#11556) 2024-08-05 14:00:32 -07:00
ActorLineageProfiler.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
AnnotateActor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
AsyncFileS3BlobStore.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
AsyncTaskThread.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
Atomic.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
AuditUtils.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
BackupAgentBase.actor.cpp New restore consolidated commit (#11901) 2025-01-22 14:54:55 -08:00
BackupContainer.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
BackupContainerFileSystem.actor.cpp New restore consolidated commit (#11901) 2025-01-22 14:54:55 -08:00
BackupContainerLocalDirectory.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
BackupContainerS3BlobStore.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
BackupTLSConfig.cpp Remove duplicate code. Move BackupTLSConfig.* from fdbbackup to 2024-12-05 08:25:25 -08:00
BlobCipher.cpp Make some codeprobes rare (#11607) 2024-08-26 22:33:38 -07:00
BlobConnectionProvider.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
BlobGranuleCommon.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
BlobGranuleFiles.cpp Make some codeprobes rare (#11607) 2024-08-26 22:33:38 -07:00
BlobGranuleReader.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
BlobMetadataUtils.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
BuildFlags.h.in More copyright end year updates (#11556) 2024-08-05 14:00:32 -07:00
BulkDumping.cpp Bulkload FDBCLI Command (#11886) 2025-01-15 09:27:59 -08:00
BulkLoading.cpp Add multiparting to s3client. (#11920) 2025-02-13 09:06:17 -08:00
CMakeLists.txt Make BulkDump work with S3 (#11822) 2024-12-18 13:29:36 -08:00
ClientKnobCollection.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
ClientKnobs.cpp New restore consolidated commit (#11901) 2025-01-22 14:54:55 -08:00
ClientStatusReport.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
ClusterConnectionFile.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
ClusterConnectionKey.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
ClusterConnectionMemoryRecord.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
CommitProxyInterface.cpp Fix typos 2024-02-07 01:16:00 +02:00
ConfigKnobs.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
ConfigTransactionInterface.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
CoordinationInterface.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
DataDistributionConfig.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
DatabaseBackupAgent.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
DatabaseConfiguration.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
DatabaseContext.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
FDBAWSCredentialsProvider.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
FDBTypes.cpp Add rocksdb, sharded rocksdb to configure workload (#11654) 2024-09-12 21:03:06 -07:00
FileBackupAgent.actor.cpp New restore consolidated commit (#11901) 2025-01-22 14:54:55 -08:00
FluentDSampleIngestor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
GlobalConfig.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
IConfigTransaction.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
IKnobCollection.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
ISingleThreadTransaction.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
IdempotencyId.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
JsonBuilder.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
KeyRangeMap.actor.cpp New restore consolidated commit (#11901) 2025-01-22 14:54:55 -08:00
LinkTest.cpp Add test executables to catch missing symbols 2022-07-06 14:49:33 -07:00
LocalClientAPI.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
ManagementAPI.actor.cpp Bulkload FDBCLI Command (#11886) 2025-01-15 09:27:59 -08:00
MetaclusterRegistration.cpp More copyright end year updates (#11556) 2024-08-05 14:00:32 -07:00
MonitorLeader.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
MultiVersionTransaction.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
MutationLogReader.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
NameLineage.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
NativeAPI.actor.cpp Database Per-Range Lock (#11693) 2024-10-23 16:25:56 -07:00
ParallelStream.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
PaxosConfigTransaction.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
RESTClient.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
RESTUtils.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
RYWIterator.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
RandomKeyValueUtils.cpp More copyright end year updates (#11556) 2024-08-05 14:00:32 -07:00
ReadYourWrites.actor.cpp Remove the usage of txsTag (#11688) 2024-09-30 07:53:37 -04:00
RestoreInterface.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
S3BlobStore.actor.cpp Add multiparting to s3client. (#11920) 2025-02-13 09:06:17 -08:00
S3Client.actor.cpp Add multiparting to s3client. (#11920) 2025-02-13 09:06:17 -08:00
S3Client_cli.actor.cpp Add multiparting to s3client. (#11920) 2025-02-13 09:06:17 -08:00
Schemas.cpp Add rocksdb version to status json. (#11868) 2025-01-09 12:56:09 -08:00
ServerKnobCollection.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
ServerKnobs.cpp Update shared rocksdb knobs. #11936 (#11938) 2025-02-11 15:41:02 -08:00
SimpleConfigTransaction.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
SpecialKeySpace.actor.cpp Refactor locality-based exclusion checks to reduce additional overhead (#11838) 2024-12-20 20:45:31 +00:00
StackLineage.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
StatusClient.actor.cpp Refactor StatusClient to have message type enum to name map 2024-11-21 11:44:46 -08:00
StorageServerInterface.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
Subspace.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
SystemData.cpp Bulkload Engine Support General Storage Engine and Fix BulkLoad Bugs (#11898) 2025-02-06 12:04:13 -08:00
TagThrottle.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
TaskBucket.actor.cpp New restore consolidated commit (#11901) 2025-01-22 14:54:55 -08:00
Tenant.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
TenantManagement.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
TestKnobCollection.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
ThreadSafeTransaction.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
Tracing.actor.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
TransactionLineage.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
Tuple.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
TupleVersionstamp.cpp Add testing that a metacluster can be used after a restore. Fix some bugs found by this related to the restore ID and tenant tombstones. 2023-06-21 08:59:22 -07:00
VersionVector.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
VersionedMap.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
WriteMap.cpp Update end year in copyright header 2024-08-02 09:40:11 -07:00
azurestorage.cmake sync with upstream main 2022-08-31 15:46:39 -07:00
notified_support.swift Add code back 2024-07-10 18:52:14 -07:00
versions.h.cmake
…
zipf.c fdbclient compiling 2022-06-27 18:31:05 -06:00