4b66645d80
Granule file performance benchmark and improvements ( #7742 )
...
* added cpu microbenchmark for blob granule files
* Added edge case read benchmarks, and sorting memory deltas
* Sorted merge for granule files
* key block comparison optimization in granule files
* More performance improvements to granule file read
* fixing zlib not supported build
* fixing formatting
* Added debug macro for new debugging prints
* review comments
* more strict compression size validation assert
2022-08-02 11:36:44 -05:00
ef6012c1d1
Encrypt BlobGranule delta files ( #7735 )
...
* Encrypt BlobGranule delta files
Description
diff-1: Address review comments
Major changes proposed by the patch are:
1. Refactor code to allow caching of 'encryption key ctx' as part of
BlobFilePointerRef. The refactoring allows snapshot and/or delta files
to store their own file encryption context.
2. Enable BlobGranule delta file encryption/decryption semantics.
Testing
BlobGranuleCorrrectness
BlobGranuleCorrectnessClean
BlobGranuleFileUnitTestToml
Description
Testing
2022-08-01 16:34:44 -07:00
c6342a6e5b
Merge branch 'main' into features/authz
2022-07-27 20:51:32 +02:00
5169616b16
Fix unresolved merge conflicts
2022-07-27 00:38:16 +02:00
0609aa57d1
Merge branch 'main' into improved_cf_testing
2022-07-26 16:44:13 -05:00
15e7a4b186
addressing review comments
2022-07-26 14:20:35 -05:00
ea9018460a
cleanup and polish
2022-07-22 15:13:32 -05:00
33053a0a7a
Bug fix and cleanup
2022-07-22 14:34:38 -05:00
095a5a4868
First version of key-sorted delta files
2022-07-22 11:43:49 -05:00
80f4c059f1
Added full granule read unit test
2022-07-21 14:14:01 -05:00
d4a2f39743
Completed delta format unit test
2022-07-21 12:40:38 -05:00
6fc0d61146
delta file test and delta generation
2022-07-21 11:49:13 -05:00
16e63578c3
Refactoring and adding to BlobGranuleFiles data generation
2022-07-21 09:47:11 -05:00
2a0381f44d
Merge pull request #7603 from sfc-gh-ahusain/ahusain-fix-test
...
Fix snapshotFormatUnitTest infinite loop
2022-07-20 14:17:39 -05:00
5540f92e2e
Refactoring randomExp to the proper place
2022-07-20 08:36:03 -05:00
3acba4e5d7
Fix snapshotFormatUnitTest infinite loop
...
Patch addresses an issue where loop generating random buffer
for the test might run in an infinite loop if KeySpace gets
exhausted but the 'targetBuffer' bytes aren't generated.
Description
Testing
2022-07-15 14:34:33 -07:00
05e7f34c80
BlobFile Encryption and compression support
...
Fix windows CI build issue.
Description
Testing
2022-07-15 11:04:02 -07:00
f6f117592d
BlobFile Encryption and compression support
...
- Limit verbose logging under DEBUG_MACRO
- Update/Add code documentation
Description
Testing
2022-07-14 17:04:14 -07:00
3a3811d6cc
BlobFile Encryption and compression support
...
Few fixes
- Update BlobGranuleCipherKeysMeta to use 'std::string' to
persist 'Initialization Vector'
- Update WriteSnapshot to persist 'BlobGranuleCipherKeysMeta' as part
of BlobFileIndex
Description
Testing
2022-07-14 17:04:14 -07:00
24b2de8de8
BlobFile Encryption and compression support
...
Description
Testing
2022-07-14 17:04:14 -07:00
b85fbaef52
Merge pull request #7395 from sfc-gh-jslocum/bg_file_chunking
...
Chunked Snapshot Files
2022-07-13 17:22:34 -05:00
0b0ac16a4c
Merge branch 'main' into granule_merging
2022-07-12 09:09:30 -05:00
9e64037b25
Merge branch 'main' into bg_file_chunking
2022-06-30 17:13:02 -05:00
d35445a868
enforce include modularization in cmake
2022-06-23 14:37:35 -06:00
29c04cf3e1
Refactored BG File structs from discussion
2022-06-17 11:24:00 -05:00
3f871f4123
Chunked Snapshot Files passes unit test
2022-06-15 14:52:28 -05:00
d6920cde28
Implemented blob granule merging
2022-06-09 10:50:53 -05:00
fcd20c479d
addressing review comments
2022-06-03 08:36:07 -05:00
458e8018e1
review comments
2022-05-25 17:16:56 -05:00
dd637dd6a7
passes tenant simulation test
2022-05-25 17:16:56 -05:00
e3a570d50d
Blob worker tenant integration mostly works
2022-05-25 17:16:56 -05:00
61474d5d54
Future-proof blob granules with full file size
2022-03-29 08:06:07 -05:00
0ff1cd102e
Merge branch 'main' into bg_client_bindings_improvements
2022-03-24 17:19:01 -05:00
3cd1e5599e
Cleanup
2022-03-24 17:15:11 -05:00
989dd8d7eb
Finished implementing beginVersion
2022-03-24 17:15:11 -05:00
1b1182f414
Blob Worker side of beginVersion done, added unit test
2022-03-24 17:15:11 -05:00
b42c49ca0a
Adding full file length for future-proofing
2022-03-24 15:56:03 -05:00
b92c6ea92c
granule parallelism, metrics, and other improvements to bg client bindings
2022-03-24 13:44:09 -05:00
a71099471b
Update copyright header dates
2022-03-21 13:36:23 -07:00
1a5bf25b5c
Update code base to use fmt 8.1.1
2022-03-04 15:52:06 -08:00
a82845af43
Merge branch 'master' into bg_bindings
2021-12-01 16:55:28 -06:00
7f4fcc8c2c
Added FDBResult and made readBlobGranules use it
2021-12-01 16:22:05 -06:00
e3839e198a
Cleanup
2021-11-03 13:41:41 -05:00
a5b2f464e3
Refactored granule materialization code to be called by c client as well as internal granule management
2021-11-03 13:19:07 -05:00
Josh Slocum