Commit Graph

137 Commits

Author SHA1 Message Date
A.J. Beamon c4b0f6eaae
Add an internal C API to support connection to a cluster using a connection string (#7438)
* Add an internal C API to support memory connection records

* Track shared state in the client using a unique and immutable cluster ID from the cluster

* Add missing code to store the clusterId in the database state object

* Update some arguments to pass by const&
2022-07-07 10:12:49 +02:00
Josh Slocum 34e6a8f942
Merge pull request #7399 from sfc-gh-jslocum/bg_tenant_improvements
Bg tenant improvements
2022-06-17 11:19:41 -05:00
Josh Slocum b3597ef3a8 Added plumbing for tenant-aware purge granules 2022-06-16 13:04:34 -05:00
Hao Fu 9cee4c94e7
Safely remove fdb_transaction_get_range_and_flat_map (#7314) 2022-06-13 19:05:00 -07:00
Hao Fu 853e6a346b
Optimization: support removing index conditionally (#7116) 2022-05-13 10:10:43 -07:00
Aaron Molitor cbaef8f03b update version to 7.2.0 2022-04-11 23:23:27 -05:00
Josh Slocum 6276cebad9
Blob integration (#6808)
* Fixing leaked stream with explicit notify failed before destructor

* better logic to prevent races in change feed fetching

* Found new race that makes assert incorrect

* handle server overloaded in initial read from fdb

* Handling more blob error types in granule retry

* Fixing rollback metadata problem, added better debugging

* Fixing version race when fetching change feed metadata

* Better racing split request handling

* fixing assert

* Handle change feed popped check in the blob worker

* fix: do not use a RYW transaction for a versionstamp because of randomize API version (#6768)

* more merge conflict issues

* Change feed destroy fixes

* Fixing change feed destroy and move race

* Check error condition in BG file req

* Using relative endpoints for blob worker interface

* Fixing bug in previous fix

* More destroy and move race fixes

* Don't update empty version on destroy in case it gets rolled back. moved() and removing will take care of ensuring it is not read

* Bug fix (#6796)

* fix: do not use a RYW transaction for a versionstamp because of randomize API version

* fix: if the initialSnapshotVersion was pruned, granule history was incorrect

* added a way to compress null bytes in printable()

* Fixing durability issue with moving and destroying change feeds

* Adding fix for not fully deleting files for a granule that child granules need to re-snapshot

* More destroy and move races

* Fixing change feed destroy and pop races

* Renaming bg prune to purge, and adding a C api and unit test for it

* more cleanup

* review comments

* Observability for granule purging

* better handling for change feed not registered

* Fixed purging bugs (#6815)

* fix: do not use a RYW transaction for a versionstamp because of randomize API version

* fix: if the initialSnapshotVersion was pruned, granule history was incorrect

* added a way to compress null bytes in printable()

* fixed a few purging bugs

Co-authored-by: Evan Tschannen <evan.tschannen@snowflake.com>
2022-04-08 14:15:25 -07:00
Tao Lin eb270d44b7 Remove unnecessary FDB_API_CHANGED 2022-04-08 10:18:16 -07:00
Tao Lin 73ed34d2fb Fixes #6793: Upgrade from 7.0.0 to 7.1.0 fails because of missing function fdb_transaction_get_mapped_range 2022-04-08 09:32:18 -07:00
Jon Fu f266695a76 clear shared state map entries on version upgrade and use currentProtocolVersion instead of piping version through function parameters 2022-03-30 15:28:55 -04:00
Jon Fu 9d0a520fe0 add ProtocolVersion to shared state and state cluster map 2022-03-30 13:28:06 -04:00
Jon Fu 53d252fda0 Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache-multi-threaded 2022-03-29 17:23:08 -04:00
Jon Fu bdbe78e48f address code review comments 2022-03-29 17:20:22 -04:00
Josh Slocum b92c6ea92c granule parallelism, metrics, and other improvements to bg client bindings 2022-03-24 13:44:09 -05:00
Jon Fu 7c3151142e Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache-multi-threaded 2022-03-23 16:41:33 -04:00
Jon Fu 4466e00716 Fix multithreaded GRV cache code and enable grv caching in multithreaded ctest suite 2022-03-23 16:09:12 -04:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Jon Fu 9731d96797 Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache-multi-threaded 2022-03-16 14:32:33 -04:00
Jon Fu 7ab205391b Revert "remove all multi-version client code related to grv cache"
This reverts commit 6d05ecffc2.
2022-03-16 14:26:36 -04:00
A.J. Beamon 8bc2b283e1 Create a special keys API to create, delete, and read tenants. Remove the C API to create/delete tenants. 2022-03-15 09:23:30 -07:00
A.J. Beamon c89b1e3537 Use TenantName type instead of StringRef in various places. Add a function on transactions to get the tenant name being used for the transaction. 2022-03-15 09:21:27 -07:00
A.J. Beamon c635dcd3ad Add tenant support in the FDB native client 2022-03-15 09:21:27 -07:00
Tao Lin e2c7c30faf
GetMappedRange support serializable & check RYW & continuation (#6181) 2022-03-10 10:05:44 -08:00
A.J. Beamon 3ae98189af Server processes send unknown_tenant responses back to clients, which is meant to be retried after updating the tenant cache. Fix bug where key selectors could be truncated after applying the tenant prefix. 2022-03-06 21:54:21 -08:00
Josh Slocum da5d3e3ae8 Added new RETURN*_ON_ERROR variants to allow catching errors in other types of functions 2021-12-10 12:47:53 -06:00
Josh Slocum c11b8a3625 Improved comments 2021-12-01 17:04:55 -06:00
Josh Slocum 7f4fcc8c2c Added FDBResult and made readBlobGranules use it 2021-12-01 16:22:05 -06:00
Josh Slocum 329091e14f Merge branch 'master' into bg_bindings 2021-11-11 10:13:37 -06:00
Josh Slocum b8ac4213a1 Switched BG APIs to transaction instead of database 2021-11-11 08:59:06 -06:00
Tao Lin fdb3b72e35 Introduce GetRangeAndFlatMap to push computations down to FDB
Re-introduce #5609
2021-11-09 13:52:28 -08:00
Tao Lin 586cc3b102
Revert "Introduce GetRangeAndFlatMap to push computations down to FDB" 2021-11-04 08:46:56 -07:00
Tao Lin 6c98e35893 Rename Hop to FlatMap 2021-11-03 13:32:01 -07:00
Tao Lin 0853661d13 Introduce getRangeAndHop to push computations down to FDB 2021-11-03 13:21:16 -07:00
Josh Slocum d6a31078fe C API for blob granules 2021-11-02 10:01:23 -05:00
sfc-gh-tclinkenbeard e62e6503ac Fix most delete-non-virtual-dtor clang warnings 2021-07-21 23:32:44 -07:00
Jingyu Zhou 3ce31cb347 Upgrade api version from 700 to 710 2021-05-05 21:15:00 -07:00
A.J. Beamon 823873a9aa Address review comments:
Use nullptr instead of NULL
Use const& for a parameter
Add some comments
2021-04-26 14:39:27 -07:00
A.J. Beamon a794fca932 Support 5.0 (and earlier) client versions by adding GRV probing for old versions. Update the C bindings implementation of get_server_protocol to convert the ProtocolVersion object into a uint64_t. Rename a misleading protocol version alias. 2021-04-23 15:00:21 -07:00
A.J. Beamon b2d6930103 The multi-version client monitors the cluster's protocol version and only activates the client library that can connect. 2021-04-15 11:45:14 -07:00
A.J. Beamon 3ed0d614d2 Move fdb_get_server_protocol to be a function on the database object. Add an argument for expected_version that can be used to signal that the function shouldn't return unless the protocol version is different. 2021-04-14 12:50:30 -07:00
Nim Wijetunga 73571ad52e Merge branch 'master' of github.com:sfc-gh-nwijetunga/foundationdb into network_busyness
* 'master' of github.com:sfc-gh-nwijetunga/foundationdb: (55 commits)
  Port lost changes from #4004
  Fix markdown
  Factor our refreshTransaction gadget
  Align FastAllocator memory to 4096 for size >= 4096
  Fix heap-use-after-free's
  Disable lsan in joshua
  Manually apply changes made to BackupContainer.actor.cpp on release-6.3 to the new files where that code is located.
  add release branch instructions
  Fix PR number in release note link
  fixed a simulation bug where a process on an unreliable machine would be considered reliable by the simulator
  Add tini as PID 1 to docker image (#4363)
  Fix typo
  Fix broken_promise bug
  Explain FastAllocator<4096> special case
  Inform LSAN that we have unaligned pointers
  apply clang-format to flow/Platform.h
  Fix typos
  Address review comments
  Change the macro that forbids exit() calls to be a static assertion
  Add versionstamp serialization information
  ...
2021-03-18 18:58:43 +00:00
Nim Wijetunga ea922aa648 address pr comments 2021-03-17 18:16:18 +00:00
Nim Wijetunga bf5f83d2bf address pr comments 2021-03-16 21:49:36 +00:00
Nim Wijetunga f0d79b3d86 Inital implementation of network busyness 2021-03-15 23:23:56 +00:00
Andrew Noyes a76c7b9754 Fix several memory leaks and a thread leak 2021-03-11 23:58:32 +00:00
Vishesh Yadav 2cd3f45fd6 Merge remote-tracking branch 'apple/release-6.3' into master-format-final
release-6.3 was recently merged, and there were two PRs which were
merged in between and got those changes in here. Hence, since all the
changes were in, discarded the incoming changes and accepted all
current.
2021-03-10 16:50:44 -08:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Scott Fines bdc888ba0a fixing formatting errors 2021-03-09 11:34:15 -06:00
Scott Fines 2db95bebf9
Merge branch 'release-6.3' into buffer_defaults 2021-03-09 08:53:12 -06:00
Scott Fines a9324ebbf6 Adjust default native buffer sizes to be more efficient 2021-03-08 13:31:44 -06:00