Commit Graph

256 Commits

Author SHA1 Message Date
Lukas Joswiak 73a7c32982
Add fdbcli command to read/write version epoch (#6480)
* Initialize cluster version at wall-clock time

Previously, new clusters would begin at version 0. After this change,
clusters will initialize at a version matching wall-clock time. Instead
of using the Unix epoch (or Windows epoch), FDB clusters will use a new
epoch, defaulting to January 1, 2010, 01:00:00+00:00. In the future,
this base epoch will be modifiable through fdbcli, allowing
administrators to advance the cluster version.

Basing the version off of time allows different FDB clusters to share
data without running into version issues.

* Send version epoch to master

* Cleanup

* Update fdbserver/storageserver.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Jump directly to expected version if possible

* Fix initial version issue on storage servers

* Add random recovery offset to start version in simulation

* Type fixes

* Disable reference time by default

Enable on a cluster using the fdbcli command `versionepoch add 0`.

* Use correct recoveryTransactionVersion when recovering

* Allow version epoch to be adjusted forwards (to decrease the version)

* Set version epoch in simulation

* Add quiet database check to ensure small version offset

* Fix initial version issue on storage servers

* Disable reference time by default

Enable on a cluster using the fdbcli command `versionepoch add 0`.

* Add fdbcli command to read/write version epoch

* Cause recovery when version epoch is set

* Handle optional version epoch key

* Add ability to clear the version epoch

This causes version advancement to revert to the old methodology whereas
versions attempt to advance by about a million versions per second,
instead of trying to match the clock.

* Update transaction access

* Modify version epoch to use microseconds instead of seconds

* Modify fdbcli version target API

Move commands from `versionepoch` to `targetversion` top level command.

* Add fdbcli tests for

* Temporarily disable targetversion cli tests

* Fix version epoch fetch issue

* Fix Arena issue

* Reduce max version jump in simulation to 1,000,000

* Rework fdbcli API

It now requires two commands to fully switch a cluster to using the
version epoch. First, enable the version epoch with `versionepoch
enable` or `versionepoch set <versionepoch>`. At this point, versions
will be given out at a faster or slower rate in an attempt to reach the
expected version. Then, run `versionepoch commit` to perform a one time
jump to the expected version. This is essentially irreversible.

* Temporarily disable old targetversion tests

* Cleanup

* Move version epoch buggify to sequencer

This will cause some issues with the QuietDatabase check for the version
offset - namely, it won't do anything, since the version epoch is not
being written to the txnStateStore in simulation. This will get fixed in
the future.

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-04-08 12:33:19 -07:00
A.J. Beamon e8a38bb048 Fix bug in tenant map range reads when the end key is outside of the tenant map range. 2022-03-25 14:24:25 -07:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
A.J. Beamon 7d9148857f Avoid promoting transactions to have system key access when using special keys 2022-03-17 08:33:53 -07:00
A.J. Beamon e8077b65e1
Merge pull request #6559 from sfc-gh-ajbeamon/fdb-tenant-client
Add client support for tenants
2022-03-15 12:40:17 -07:00
Jingyu Zhou e89ee7d5a0
Merge pull request #6589 from sfc-gh-tclinkenbeard/fix-typos
Fix typos
2022-03-15 10:10:23 -07:00
A.J. Beamon 8a5107af03 Fix a couple variable initialization issues in the tenant MVC implementation. Encode tenant prefixes with unicode escape sequences for the list command. 2022-03-15 09:23:30 -07:00
A.J. Beamon 8bc2b283e1 Create a special keys API to create, delete, and read tenants. Remove the C API to create/delete tenants. 2022-03-15 09:23:30 -07:00
A.J. Beamon c635dcd3ad Add tenant support in the FDB native client 2022-03-15 09:21:27 -07:00
sfc-gh-tclinkenbeard 8dcac2f76d Fix typos 2022-03-13 10:02:11 -03:00
Ben Collins 32f2731773
Fix static initialization order for maxAllowedVersion (#6570)
* Fix static initialization order for maxAllowedVersion

As it was defined, maxAllowedVerion was statically and globally initialized with a dependency on CLIENT_KNOBS. Initialization order was not spelled out and therefore this relied on chance specifics of order. In some cases these two were constructed in the wrong order and led to a segfault on startup. Fix by deferring initialization of maxAllowedVerion.

* Fix formatting with clang-format
2022-03-11 11:02:35 -08:00
A.J. Beamon 36435af4f5 Add support for a limits hint in special keys. Fix a typo and resolve some comment formatting weirdness. 2022-03-06 21:52:39 -08:00
A.J. Beamon 41d1e4b5d5 Fix several instances where we were converting keys, ranges, and key selectors into strings before passing them into trace events. This had the effect of double escaping certain characters. 2022-03-04 13:58:36 -08:00
Renxuan Wang 06b1d06d38 Support hostname in coordinators commands. 2022-02-24 23:02:29 -08:00
Renxuan Wang 70d723f70c Move the construction of ConnectionString in changeQuorumChecker() to coordinatorsCommitActor(). 2022-02-24 23:02:29 -08:00
A.J. Beamon 2eda48416c Undo some changes from LOCK_AWARE to READ_LOCK_AWARE. Apparently we have some dependencies in composed functions where a read-only function is setting lock awareness to the benefit of another function that does writes. 2022-01-28 10:55:02 -08:00
A.J. Beamon 8418cb2839 Mark various externally exposed system transactions as read/access system keys. Also adjust a few lock aware declarations to read lock aware. 2022-01-27 13:58:33 -08:00
A.J. Beamon d8e161f89e Refactor NativeAPI transactions to create and pass around a reference counted state object. Watches no longer use the tranasction info object but instead use their own state. 2021-12-17 11:57:39 -08:00
sfc-gh-tclinkenbeard ebcc023b6f Enable missing-field-initializers clang warning 2021-11-01 14:18:31 -07:00
A.J. Beamon e882eb33fc Abstract the cluster file into a cluster connection record that can be backed by something other than the filesystem. 2021-10-22 11:05:18 -07:00
Vaidas Gasiunas 16171d8252 Refactoring well-known endpoint registration
- List all well-known endpoints of FDB in a single enum
- Identify well-known endpoints by plain IDs
2021-09-21 11:05:31 -06:00
Xiaoge Su abf73047ca Enforce std:: specifier rather than using namespace 2021-09-16 19:40:28 -07:00
Chaoguang Lin e7d3516084 solve comments 2021-09-14 11:04:02 -07:00
Chaoguang Lin 1b3bae899b update comments 2021-09-14 11:04:02 -07:00
Chaoguang Lin 5c5a5d249e Update auto_coordinators special key error message 2021-09-14 11:04:02 -07:00
Chaoguang Lin cb9dde358c Solve comments, refatctor the error handling part 2021-09-13 13:28:48 -07:00
Chaoguang Lin 2aa0a5cf58 The lock special key has the lock UID as the value 2021-09-03 12:15:56 -07:00
FDB Formatster 2c788c233d apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-08-27 17:07:47 -07:00
Mohamed Oulmahdi 164ce4b334 Add Windows specific code for system time 2021-08-18 18:06:36 +02:00
Lukas Joswiak a605fb3852
Merge pull request #5026 from sfc-gh-ljoswiak/fixes/alp6
Actor sampling
2021-08-11 13:44:17 -07:00
Steve Atherton 2bf7b9b7a9 Remove many calls to .toString(), .printable(), and printable(...) from TraceEvent .detail() values to avoid double-escaping characters such that the same values look different in different events and can't be easily searched for. Also fixed a few warnings found by clang. 2021-08-08 00:04:54 -07:00
Lukas Joswiak 5dc9a97230 Merge branch 'master' into fixes/alp6 2021-08-01 20:42:52 -07:00
Lukas Joswiak 59d535149e Merge branch 'master' into fixes/alp6 2021-07-27 10:07:18 -07:00
sfc-gh-tclinkenbeard 3442ebd3b7 Fix more -Wreorder-ctor warnings across many files 2021-07-24 11:20:51 -07:00
Steve Atherton f596a81073 Rename ::TRUE and ::FALSE in BooleanParams to ::True and ::False so as to not conflict with the TRUE and FALSE macros provided by the Windows and MacOS SDKs. 2021-07-17 00:11:40 -07:00
sfc-gh-tclinkenbeard 8cc40e3a2b Expand use of BOOLEAN_PARAM 2021-07-02 21:41:50 -07:00
Neethu Haneesha Bingi decfb610ff Minor review comments. 2021-06-25 15:04:49 -07:00
Neethu Haneesha Bingi 66f2518405 exclude to work with any locality data match. 2021-06-23 18:03:27 -07:00
Neethu Haneesha Bingi cbe714acd0 Status json schema update, includelocalities back for consistency check, review comments. 2021-06-23 18:03:27 -07:00
Neethu Haneesha Bingi 4ad5926a25 Snake naming of keys and added comments to all new functions. 2021-06-23 18:03:27 -07:00
Neethu Haneesha Bingi 73752f441b exclude locality:clang-format, ranged loops, documentation, tracking addStoragesever for exclusion. 2021-06-23 18:03:27 -07:00
Neethu Haneesha Bingi 62355571d0 exclude servers based on locality match 2021-06-23 18:03:27 -07:00
Lukas Joswiak ca79b8eaab Various ALP fixes 2021-06-04 15:01:18 -07:00
Lukas Joswiak 153de33f57 Revert "Merge pull request #4802 from sfc-gh-ljoswiak/revert/actor-lineage"
This reverts commit 6499fa178e, reversing
changes made to 1512631957.
2021-06-04 13:31:55 -07:00
Lukas Joswiak ba25b95c6a Fix global config not updating on server processes 2021-06-03 14:27:24 -07:00
Chaoguang Lin 5262e41098 Grab moveKeysLockOwnerKey and moveKeysLockWriteKey when update dd related system keys 2021-05-27 17:25:08 +00:00
Lukas Joswiak 4ea760b2a9 Revert "Merge pull request #4136 from sfc-gh-mpilman/features/actor-lineage"
This reverts commit da41534618, reversing
changes made to e6300905d6.
2021-05-10 20:26:12 -07:00
Andrew Noyes ed464d3514 Deduplicate 2021-05-05 16:30:59 +00:00
Andrew Noyes 946e3f989f Use a deterministic iteration order for committing in the special key space 2021-05-05 16:30:59 +00:00
sfc-gh-tclinkenbeard 5c2d7b6080 Create RangeResult type alias 2021-05-03 13:14:16 -07:00