Commit Graph

382 Commits

Author SHA1 Message Date
Junhyun Shim 78ea2b40b6 Add assertion to check g_currentDelivery* is unset before delivery 2022-07-28 01:16:42 +02:00
Junhyun Shim e2a3fedfc7
Merge branch 'main' into features/authz 2022-07-27 00:08:57 +02:00
Markus Pilman 1de37afd52
Make TEST macros C++ only (#7558)
* proof of concept

* use code-probe instead of test

* code probe working on gcc

* code probe implemented

* renamed TestProbe to CodeProbe

* fixed refactoring typo

* support filtered output

* print probes at end of simulation

* fix missed probes print

* fix deduplication

* Fix refactoring issues

* revert bad refactor

* make sure file paths are relative

* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
Junhyun Shim ac5436a090 Catch up on changes to main 2022-07-12 16:48:09 +02:00
Junhyun Shim 545a9a8043 Fix bugs and add token timeout-in-cache test 2022-07-11 16:58:04 +02:00
Markus Pilman 56541167c2 send token with tenant info object 2022-07-06 20:58:17 -06:00
Markus Pilman 624430b6ae addess review comments 2022-06-27 14:22:23 -06:00
Markus Pilman 13d8b13722 Migrated Authz code to use JWT 2022-06-13 18:20:27 -06:00
Markus Pilman 799fe32346 Merge remote-tracking branch 'origin/main' into features/authz 2022-06-13 18:02:11 -06:00
Josh Slocum 5621fcff40 formatting fix 2022-05-25 17:16:56 -05:00
Josh Slocum fd7b662767 cleaning up 2022-05-25 17:16:56 -05:00
Josh Slocum e3a570d50d Blob worker tenant integration mostly works 2022-05-25 17:16:56 -05:00
A.J. Beamon 993a79b42c
Restart protocol monitoring logic when we cycle the coordinator, even if its the same coordinator. Add some defensive code in case we fetch the protocol version for an absent peer. (#6397) 2022-05-19 10:00:23 +02:00
Vaidas Gasiunas b55863d958
Merge pull request #7132 from sfc-gh-vgasiunas/vgasiunas-fix-version-check-for-downgrades
Fixing the problem with client getting stuck on server downgrades
2022-05-13 10:53:18 +02:00
Andrew Noyes a92ef37d44 Log a backtrace before throwing serialization_failed 2022-05-12 20:08:36 -07:00
Vaidas Gasiunas 2f67637701 Fixing the problem with client getting stuck on server downgrades:
- Restoring the original check for strict protocol compatibility before sending packets
 - Resetting the compatibility flag when connection is closed, so that the protocol compatibility is checked again for a new connection
2022-05-11 19:12:38 +02:00
A.J. Beamon 0597ae7513 When an incompatible connection is closed, clear the state that prevents us from sending messages to it 2022-05-10 15:29:36 -07:00
Ray Jenkins dc9e782ccc
OpenTelemetry Tracing Perf Fixes (#6990) 2022-05-02 14:56:51 -05:00
Markus Pilman 145f3d15bc implemented token caching 2022-04-26 10:21:22 -06:00
Markus Pilman dc65b7e1c0 fix gcc build error, rename tenantcycle workload 2022-04-25 14:49:48 -06:00
Markus Pilman bf2a723889 fix formatting 2022-04-25 09:03:12 -06:00
Markus Pilman ccf97eb187 Make transport work 2022-04-22 17:06:05 -06:00
Markus Pilman 1da1f8cc0f Fix memory issue 2022-04-21 14:12:16 -06:00
Markus Pilman b61c9ab329 Merge remote-tracking branch 'origin/main' into features/authz 2022-04-20 14:40:14 -06:00
Binglin Chang 408c0cf1c9
Fix compile errors on ubuntu 20.04 (#4931) 2022-04-20 10:00:46 -07:00
Markus Pilman 436d15ea98 Implemented token management and testing
Code is still untested at this point, but testing code was written

Implemented token management and testing

Code is still untested at this point, but testing code was written
2022-04-16 11:29:13 -06:00
Markus Pilman 577f0dca99 Make context variables type safe 2022-04-14 11:24:19 -06:00
Markus Pilman 420a901938 Added verification logic 2022-04-13 12:30:18 -06:00
Markus Pilman f9eca3884b implemented base logic 2022-04-12 12:02:43 -06:00
Markus Pilman bf956f5630 Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-04-07 13:29:27 -06:00
Markus Pilman e2d7d4075d multiple bug fixes 2022-04-07 11:08:07 -06:00
Chaoguang Lin 7d365bd1bb
Remote ikvs debugging (#6465)
* initial structure for remote IKVS server

* moved struct to .h file, added new files to CMakeList

* happy path implementation, connection error when testing

* saved minor local change

* changed tracing to debug

* fixed onClosed and getError being called before init is finished

* fix spawn process bug, now use absolute path

* added server knob to set ikvs process port number

* added server knob for remote/local kv store

* implement simulator remote process spawning

* fixed bug for simulator timeout

* commit all changes

* removed print lines in trace

* added FlowProcess implementation by Markus

* initial debug of FlowProcess, stuck at parent sending OpenKVStoreRequest to child

* temporary fix for process factory throwing segfault on create

* specify public address in command

* change remote kv store knob to false for jenkins build

* made port 0 open random unused port

* change remote store knob to true for benchmark

* set listening port to randomly opened port

* added print lines for jenkins run open kv store timeout debug

* removed most tracing and print lines

* removed tutorial changes

* update handleIOErrors error handling to handle remote-ikvs cases

* Push all debugging changes

* A version where worker bug exists

* A version where restarting tests fail

* Use both the name and the port to determine the child process

* Remove unnecessary update on local address

* Disable remote-kvs for DiskFailureCycle test

* A version where restarting stuck

* A version where most restarting tests green

* Reset connection with child process explicitly

* Remove change on unnecessary files

* Unify flags from _ to -

* fix merging unexpected changes

* fix trac.error to .errorUnsuppressed

* Add license header

* Remove unnecessary header in FlowProcess.actor.cpp

* Fix Windows build

* Fix Windows build, add missing ;

* Fix a stupid bug caused by code dropped by code merging

* Disable remote kvs by default

* Pass the conn_file path to the flow process, though not needed, but the buildNetwork is difficult to tune

* serialization change on readrange

* Update traces

* Refactor the RemoteIKVS interface

* Format files

* Update sim2 interface to not clog connections between parent and child processes in simulation

* Update comments; remove debugging symbols; Add error handling for remote_kvs_cancelled

* Add comments, format files

* Change method name from isBuggifyDisabled to isStableConnection; Decrease(0.1x) latency for stable connections

* Commit the IConnection interface change, forgot in previous commit

* Fix the issue that onClosed request is cancelled by ActorCollection

* Enable the remote kv store knob

* Remove FlowProcess.actor.cpp and move functions to RemoteIKeyValueStore.actor.cpp; Add remote kv store delay to avoid race; Bind the child process to die with parent process

* Fix the bug where one process starts storage server more than once

* Add a please_reboot_remote_kv_store error to restart the storage server worker if remote kvs died abnormally

* Remove unreachable code path and add comments

* Clang format the code

* Fix a simple wait error

* Clang format after merging the main branch

* Testing mixed mode in simulation if remote_kvs knob is enabled, setting the default to false

* Disable remote kvs for PhysicalShardMove which is for RocksDB

* Cleanup #include orders, remove debugging traces

* Revert the reorder in fdbserver.actor.cpp, which fails the gcc build

Co-authored-by: “Lincoln <“lincoln.xiao@snowflake.com”>
2022-03-31 17:08:59 -07:00
Markus Pilman b595d4462f Throw error on unauthorized access 2022-03-29 14:58:43 -06:00
Josh Slocum f27475e2f4 Merge branch 'main' into blob_integration 2022-03-22 11:41:58 -05:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Markus Pilman 1fbeca8038 fix memory issue 2022-03-19 11:03:32 +01:00
Josh Slocum 37e7c80f26 Merge branch 'main' into blob_integration 2022-03-17 18:45:42 -05:00
Evan Tschannen 7908ea54f3 fix: NetNotifiedQueueWithAcknowledgements could miss disconnects that happen during the delay(0) in deliver() 2022-03-17 09:57:46 -07:00
Markus Pilman 158da462f7 fix memory bug 2022-03-17 14:37:05 +01:00
Markus Pilman 118b53b7cf Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-03-17 12:06:44 +01:00
Markus Pilman bed799220a Addressed review comments, added test 2022-03-15 16:57:26 +01:00
sfc-gh-tclinkenbeard baec03090e Fix "guarantee" misspelling 2022-03-14 16:33:09 -07:00
sfc-gh-tclinkenbeard 8dcac2f76d Fix typos 2022-03-13 10:02:11 -03:00
Josh Slocum a2bbb188c3 Fixing delayed health updater to mark address failed when too many connections triggers before new connection starts, instead of after 2022-03-11 16:50:38 -06:00
Markus Pilman 8fac0081a8 Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-03-09 11:00:00 +01:00
Markus Pilman 0aa7823622 Pass object as reference 2022-02-25 14:54:52 +01:00
A.J. Beamon 250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
Markus Pilman 20bf3e1599 Address review coomments 2022-02-23 19:02:29 +01:00
Markus Pilman cf31e14904 Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-02-23 10:29:32 +01:00
Markus Pilman 102169ba33 Ran clang-format 2022-02-23 10:23:27 +01:00