CommitDebug trace events are useful for measuring, in detail, the time spent in
the various parts of a single transaction. Like all log events, they have a time
associated with them. This time comes for now(), which in a real fdb system is
only updated in the run loop. This renders the timestamps inaccurate in certain
CPU bound sections which don't have a wait, e.g. in the resolver.
The desired solution is to directly call ::timer_monotonic() instead of now()
for a timestamp, since ::timer_monotonic() is what now() ultimately uses in
the run loop, but is updated between waits as well.
In order to not disrupt any current uses of the logs which might depend on the
current behavior of the Time field, we introduced the `MonotonicTime` field.
This change only affects CommitDebug trace events, not trace events in general.
CommitDebug trace events are useful for measuring, in detail, the time spent in
the various parts of a single transaction. Like all log events, they have a time
associated with them. This time comes for now(), which in a real fdb system is
only updated in the run loop. This renders the timestamps inaccurate in certain
CPU bound sections which don't have a wait, e.g. in the resolver.
We want to preserve the current behavior in simulation, where the timestamps are
artificial, deterministic between runs, and only updated in the run loop.
In a real system, we prefer to use a real clock so we can use the difference
between two trace events in the logs as a measurement of elapsed time.
This does not modify the behavior of other parts of the system, which use the
cached now() for various purposes.
* client_config_tester: use a generic mechanism to set specific network options
* trace_initialize_on_setup option to initialize client traces on network setup without local IP address
* trace_initialize_on_setup: Addressing review comments
* Restore correct formatting
* trace_initialize_on_setup: Update go bindings
* Include PID for identification into trace file names by default
* Use the same naming pattern for trace files in all configurations
* Empty commit
* Upgrade tests: dump thread call stacks of the tester process if it fails to terminate
* ApiTester: log before and after stopping the network thread
* Catch and print exceptions in closeTraceFile; Close trace file at the end of MVC runNetwork
* Change trace event name for MVC runNetwork termination
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
* Update network address in trace logs; Add system monitor for flowprocess
* Create a new trace file with the correct process address for flowprocess
* Remove unused debugging traces
* Add a new error lock_file_failure; Change please_reboot_remote_kv_store to please_reboot_kv_store; Add the code to only reboot the kv store but not the worker; Remove some unnecessay traces
* Add error handling for file_not_found in handleIOErrors
* Format worker.actor.cpp file
Currently, a std::string is copied unnecessarily for every key and value
in a trace event.
This actually showed up in a jemalloc heap profile while I was
investigating something unrelated. I was surprised to see it since these
allocations should have a very short lifetime.
also in order to continue tracing the pending network thread activity.
Poll event throttler only in the network thread in order to avoid a race condition.
* add storagemetadata
* add StorageWiggler;
* fix serverMetadataKey bug
* add metadata tracker in storage tracker
* finish StorageWiggler
* update next storage ID
* change pid to server id
* write metadata when seed SS
* add status json fields
* remove pid based ppw iteration
* fix time expression
* fix tss metadata nonexistence; fix transaction retry when retrieving metadata
* fix checkMetadata bug when store type is wrong
* fix remove storage status json
* format code
* refactor updateNextWigglingStoragePID
* seperate storage metadata tracker and store type tracker
* rename pid
* wiggler stats
* fix completion between waitServerListChange and storageRecruiter
* solve review comments
* rename system key
* fix database lock timeout by adding lock_aware
* format code
* status json
* resolve code format/naming comments
* delete expireNow; change PerpetualStorageWiggleID's value to KeyBackedObjectMap<UID, StorageWiggleValue>
* fix omit start rount
* format code
* status json reset
* solve status json format
* improve status json latency; replace binarywriter/reader to objectwriter/reader; refactor storagewigglerstats transactions
* status timestamp
Dan Adkins