Commit Graph

8 Commits

Author SHA1 Message Date
Xiaoxi Wang bbcb3cc018 extract KeyBackedConfig, StorageWiggleData class; solve template resolution problem; solve MV txn and native api conflict by splitting RunTransaction file 2023-01-02 23:34:39 -08:00
Andrew Noyes dd0036f09c
Automatically clean old idempotency ids (#9039)
* Add cleanIdempotencyIds

Delete zero or more idempotency ids older than minAgeSeconds

* Automatically clean idempotency ids from first proxy

* Add test for cleaner

* Fix formatting

* Address review comments
2022-12-14 14:24:24 -08:00
Andrew Noyes 0a15f081a1
Proactively clean up idempotency ids for successful commits (#8578)
* Proactively clean up idempotency ids for successful commits

This change also includes some minor changes from my branch working on
an idempotency ids cleaner, that I'd like to get merged sooner rather
than later.

- Adding a timestamp to idempotency values
- Making IdempotencyId an actor file
- Adding commit_unknown_result_fatal
- Checking idempotencyIdsExpiredVersion in determineCommitStatus
- Some testing QOL changes

* Factor out decodeIdempotencyKey logic

* Fix formatting

* Update flow/include/flow/error_definitions.h

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Use KeyBackedObjectProperty for idempotencyIdsExpiredVersion

* Add IDEMPOTENCY_ID_IN_MEMORY_LIFETIME knob

* Rename ExpireIdempotencyKeyValuePairRequest

Also add a code probe for the case where an ExpireIdempotencyIdRequest is
received before the count is known, and add an assert

* Fix formatting and add TODO for nwijetunga

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-10-28 09:07:54 -07:00
Markus Pilman 49f0cf5ab0 Force name and description of workloads to be the same 2022-10-13 20:53:48 -06:00
Andrew Noyes 9e7630869e Test ids > 16 bytes in workload 2022-10-11 13:46:40 -07:00
Andrew Noyes e51d2a765c Make sure ids are unique 2022-10-11 13:46:40 -07:00
Andrew Noyes 43a72ad406 Use versionstamped keys instead of atomic add in test
This makes it easier to debug if something goes wrong
2022-10-11 13:46:39 -07:00
Andrew Noyes 6ef8d7ec95 Add AutomaticIdempotencyWorkload 2022-10-11 13:46:39 -07:00