Commit Graph

11 Commits

Author SHA1 Message Date
Akanksha Mahajan 96f56bb29a
Fix read-side overflow and simplify append API for large snapshot manifests (#13691)
* Fix read-side overflow and simplify append API for large snapshot manifests

  Follow-up to #13349, which fixed the write-side overflow when a snapshot manifest exceeds ~2 GB but left the read side and the API untidy. This PR addresses both.

  ### Changes

  **Simpler append API**
  There were two `append()` methods — one taking `int`, one `size_t` — and which ran depended on the argument type, which is easy to get wrong. Replaced with a single
  public `append()` that safely chunks any size, plus a clearly-named backend hook `appendImpl()` that each storage backend implements. No more overload ambiguity.

  **Read side fix**
  `readKeyspaceSnapshot` read the manifest into a buffer whose length is an `int`, so a manifest larger than 2 GB could truncate and crash on restore. It now reads into a
  `std::string` (which can exceed 2 GB) in chunks, matching the write side, and drops a redundant full copy of the manifest.

  **Knob rename**
  `BACKUP_MANIFEST_WRITE_CHUNK_SIZE` → `BACKUP_MANIFEST_CHUNK_SIZE`, since it now controls chunk size for both reads and writes.

  **Test**
  Added a unit test that reads a manifest back in many small chunks and verifies all range files and key ranges round-trip correctly.

  ### Notes
  - Range and log files are unaffected — they're already streamed in small blocks on both read and write.

* Addressed comments

* Fix clang tidy errors
2026-07-20 12:39:15 -07:00
Trevor Clinkenbeard fe42e6fd79 Enforce clang-tidy braces around long statements 2026-06-22 23:01:11 -07:00
Trevor Clinkenbeard b46c58415c Replace BUGGIFY macros with inline function 2026-05-20 14:54:51 -07:00
Trevor Clinkenbeard f4e8de2410 Preserve base reference types in mapped lambdas 2026-05-09 13:24:20 -07:00
Trevor Clinkenbeard d7f1bdc66f Prefer makeReference for polymorphic references 2026-05-09 13:13:05 -07:00
Akanksha Mahajan 54795dcc94
Design changes to pass encryption_block_size to fdbbackup command and remove knob (#13023) 2026-04-30 13:32:14 -07:00
Trevor Clinkenbeard c127105f57 Improve header file encapsulation 2026-04-24 16:03:07 +00:00
Trevor Clinkenbeard e38baf18ea Fix more clang-tidy warnings 2026-04-15 14:23:33 +00:00
Trevor Clinkenbeard bf00cbf1c0 Rename *.actor.* files 2026-04-12 22:41:05 +00:00
Trevor Clinkenbeard 695526aa48 Rename non-actor files 2026-03-22 03:06:37 +00:00
Trevor Clinkenbeard 839df4a5c5 Rename newly converted non-actor files 2026-03-20 10:42:05 +00:00