- More robustly run processes in the background
- Detect and use pypy if available
- Add a --latency_profile flag to control whether to run the latency profiling stuff
Current latency profiles have their tails dominated by writing latency
logs, which is hugely undesirable.
Now when a thread log fills up, push it to a background thread to write
to disk. At shutdown, wait for all latency traces to be flushed.