Description
The current code supports two modes for authentication encryption:
1. Single auth-token mode
2. Multi auth-token mode
The code currently uses SingleAuthToken mode only, the multi-auth
token mode was added to support large file (such as backup files)
encryption assisting encryption authentication by allowing reading
header independently from encrypted payload itself. However, the
backup files are organized as 'chunks' and every chunk is encrypted,
hence, MultiToken mode isn't used in the code.
Removing the usage saves 32 bytes per encryption header, which might
translate to decent storage saving, further, allows lesser encryption
header overhead when encrypting small Key-Value mutations.
Testing
BlobCipher unittests
EncryptionOps.toml
* Introduce "default encryption domain"
Description
In current FDB native encryption data at-rest implementation,
an entity getting encrypted (mutation, KV and/or file) is categorized
into one of following encryption domains:
1. Tenant domain, where, Encryption domain == Tenant boundaries
2. FDB system keyspace - FDB metadata encryption domain
3. FDB Encryption Header domain - used to generate digest for
plaintext EncryptionHeader.
The scheme doesn't support encryption if an entity can't be categorized
into any of above mentioned encryption domains, for instance, non-tenant
mutations are NOT supported.
Patch extend the encryption support for mutations for which corresponding
Tenant information can't be obtained (Key length shorter than TenantPrefix)
and/or mutations do not belong to any valid Tenant
(FDB management cluster data) by mapping such mutations to a
"default encryption domain".
TODO
CommitProxy driven TLog encryption implementation requires every transaction
mutation to contain 1 KV, not crossing Tenant-boundaries. Only exception to
this rule is ClearRange mutations. For now ClearRange mutations are mapped
to 'default encryption domain', in subsequent patch appropriate handling
for ClearRange mutations shall be proposed.
Testing
devRunCorrectness - 100k
Adding the following metrics:
* BlobCipherKeyCache hit/miss
* EKP: KMS requests latencies
* For each component that using encryption, they now need to pass a UsageType enum to the encryption helper methods (GetEncryptCipherKeys/GetLatestEncryptCipherKey/encrypt/decrypt) and those methods will help to log get cipher key latency samples and encryption/decryption cpu times accordingly.
Description
FDB native encryption data at-rest supports two type of cipher-keys
in-memory caching:
1. Revocable keys - with a definite expiry (future timestamp)
2. Non-revocable keys - with or without expiry timestamp and/or
refreshAt timestamp.
Patch update BlobCipherKey in-memory cache to respect EKP/KMS
supplied 'refreshAt' and 'expireAt' timestamp. GetLatestCipher
validates `cipher key freshness' as well as GetCipherKey checks
for 'cipher key liveness' before replying details to the caller.
Patch also optimizes the BlobCipher module logging by taking
following measures:
1. BLOB_CIPHER_DEBUG macro to guard spammy log messages needed
mostly for debugging failures.
2. Minimize log volume by logging cipherKey details for any new
key added to the cache, key-refreshes are not logged.
3. Categorize logs into: debug, info and warn on per-usecase basis
Testing
devRunCorrectness - 100K
EncryptOps.toml - 100K
Adding GetEncryptCipherKeys and GetLatestCipherKeys helper actors, which encapsulate cipher key fetch logic: getting cipher keys from local BlobCipherKeyCache, and on cache miss fetch from EKP (encrypt key proxy). These helper actors also handles the case if EKP get shutdown in the middle, they listen on ServerDBInfo to wait for new EKP start and send new request there instead.
The PR also have other misc changes:
* EKP is by default started in simulation regardless of. ENABLE_ENCRYPTION knob, so that in restart tests, if ENABLE_ENCRYPTION is switch from on to off after restart, encrypted data will still be able to be read.
* API tweaks for BlobCipher
* Adding a ENABLE_TLOG_ENCRYPTION knob which will be used in later PRs. The knob should normally be consistent with ENABLE_ENCRYPTION knob, but could be used to disable TLog encryption alone.
This PR is split out from #6942.
* Enable debugId tracing for encryption requests
Description
diff-1: Minor fixes, address review comment
Proposed changes include:
1. Update EncryptKeyProxy API to embded Optional<UID> for debugging
request execution.
2. Encryption participant FDB processes can set 'debugId' enabling
tracing requests within FDB cluster processes and beyond.
3. The 'debugId' if available is embedded as part of 'request_json_payload'
by RESTKmsConnector, enabling tracing request between FDB <--> KMS.
4. Fix EncryptKeyProxyTest which got broken due to recent changes.
Testing
Updated following test:
1. EncryptKeyProxy simulation test.
2. RESTKmsConnector simulation test.
Description
Testing
Description
Major changes proposed:
1. This reverts commit f38b2e8209.
2. Also add fix for Valgrind failure due to unintialized variables.
3. Improve checks to catch is cipherKey details cached in BlobCipherKeyCache
isn't as expected
Testing
Overall correctness: 10K (20220421-193911-ahusain-foundationdb-a730e5cb38541e20)
EncyrptionOps correctness: 100K (20220421-194315-ahusain-foundationdb-29c598a8b9420430)
EncryptionOps Valgrind: 100 (20220421-194434-ahusain-foundationdb-7fc5f98eddc0921a)
* Update 'salt' details for EncryptHeader AuthToken details
Description
Major changes:
1. Add 'salt' to BlobCipherEncryptHeader::cipherHeaderDetails.
2. During decryption it is possible that BlobKeyCacheId doesn't
contain required baseCipherDetails. Add API to KeyCache to
allowing re-populating of CipherDetails with a given 'salt'
3. Update BaseCipherKeyIdCache indexing using {BaseCipherKeyId, salt}
tuple. FDB processes leverage BlobCipherKeyCache to implement
in-memory caching of cipherKeys, given EncryptKeyProxy supplies
BaseCipher details, each encryption participant service would
generate its derived key by using different 'salt'. Further,
it is possible to cache multiple {baseCipherKeyId, salt} tuples;
for instance: CP encrypted mutations being deciphered by
StorageServer etc.
Testing
1. Update EncyrptionOps simulation test to simulate KeyCache miss
2. Update BlobCipher unit tests to validate above mentioned changes
* EncryptKeyProxy server APIs for simulation runs.
Description
diff-2: FlowSingleton util class
Bug fixes
diff-1: Expected errors returned to the caller
Major changes proposed are:
1. EncryptKeyProxy server APIs:
1.1. Lookup Cipher details via BaseCipherId
1.2. Lookup latest Cipher details via encryption domainId.
2. EncyrptKeyProxy implements caches indexed by: baseCipherId &
encyrptDomainId
3. Periodic task to refresh domainId indexed cache to support
'limiting cipher lifetime' abilities if supported by
external KMS solutions.
Testing
EncyrptKeyProxyTest workload to validate the newly added code.
* Encryption header authentication tokens
Description
diff-1: Allow NONE AuthTokenMode operations
Address review comments
Major changes proposed are:
1.Encryption header support two modes of generation 'authentication tokens':
a) SingleAuthTokenMode: the scheme generates single crypto-secure auth
token to protect {cipherText + header} payload. Scheme is geared towards
optimizing cost due to crypto-secure auth-token generation, however,
on decryption client needs to be read 'header' + 'encrypted-buffer'
to validate the 'auth-token'. The scheme is ideal for usecases where
payload represented by the encryptionHeader is not large and it is
desirable to minimize CPU/latency penalty due to crypto-secure ops,
such as: CommitProxies encrypted inline transactions,
StorageServer encrypting pages etc.
b) MultiAuthTokenMode: Scheme generates separate authTokens for
'encrypted buffer' & 'encryption-header'. The scheme is ideal where
payload represented by encryptionHeader is large enough such that it
is desirable to optimize cost of upfront reading full 'encrypted buffer',
compared to reading only encryptionHeader and ensuring its sanity;
for instance: backup-files
2. Leverage full crypto-secure digest as 'authentication token'
Testing
Update EncryptionOps simulation test
Update BlobCipher unit test
20220408-182229-ahusain-foundationdb-7fd2e4b19328cd44
20220408-175754-ahusain-foundationdb-5352e37e1dcabfc8
* Introduce BlobCipher interface and cipher caching interface
diff-3: Update the code to avoid deriving encryption key periodically.
Implement EncyrptBuf interface to limit memcpys.
Improve both unit test and simulation to better code coverage.
diff-2: Add specific error code for OpenSSL AES call failures
diff-1: Update encryption scheme to AES-256-CTR. Minor
updates to Header to capture more information.
Major changes proposed are:
1. Introduce encyrption header format.
2. Introduce a BlobCipher cipher key representation encoding
following information: baseCipher details, derived encryption cipher
details, creationTime and random salt.
3. Introduce interface to support block cipher encrytion and decrytion
operations. Encyrption populates encryption header allowing client to
persist them on-disk, this header is then read allowing decryption
on reads.
4. Introduce interface to allow in-memory caching of cipher keys. The
cache allowing mapping of "encryption domain" -> "base cipher id" ->
"derived cipher keys" (3D hash map). This cache interface will be used
by FDB processes participating in encryption to cache recently used
ciphers (performance optimization).
Testing:
1. Unit test to validate caching interface.
2. Update EncryptionOps simulation test to validate block cipher
operations.
* Upgrade AES 128 GCM -> AES 256, StreamCipher code refactor
Major changes proposed are:
1. Refactor StreamCipher code to enable instantiation of
multiple encryption keys. However, code still retains
a globalEncryption key semantics used in Backup file
encryption usecase.
2. Enhance StreamCipher to provide HMAC signature digest
generation. Further, the class implements HMAC encryption
key derivation function.
3. Upgrade StreamCipher to use AES 256 GCM mode from currently
supported AES 128 GCM mode.
Note: The code changes the encryption key size, however, the
feature is NOT currently in use, hence, should be OK.
3. Add EncryptionOps validation and benchmark toml supported
workload, it does the following:
a. Allow user to configure encrypt-decrypt of a fixed size
buffer or variable size buffer [100, 512K]
b. Allow user to configure number of interactions of the runs,
in each iteration: generate random data, derive an encryption
key using HMAC SHA256 method, encrypt data and
then decrypt data. It collects following metrics:
i) time taken to derive encryption key.
ii) time taken to encrypt the buffer.
iii) time taken to decrypt the buffer.
iv) total bytes encrypted and/or decrypted
c. Along with stats it basic basic validations on the encrypted
and decrypted buffer
d. On completion for test, records the above mentioned metrics
in trace files.
Markus Pilman