Commit Graph

1532 Commits

Author SHA1 Message Date
Andrew Noyes 23de546586 Only build libcoro if COROUTINE_IMPL == libcoro
It doesn't build at all on mac m1
2021-12-03 16:34:35 -08:00
sfc-gh-tclinkenbeard ec64890ac1 Remove some usages of PRId64 by using fmt library 2021-11-30 23:35:36 -08:00
Andrew Noyes b6fd402a3c Add option to use boost or libcoro
By default, use boost everywhere except windows and linux x86 (for
performance reasons)
2021-11-29 13:14:15 -08:00
Renxuan Wang 46d17d748f Change member variable fromHostname to type bool.
So that it can be serialized.
2021-11-23 14:25:02 -10:00
Evan Tschannen 37c9a1320c added --print_sim_time to print simulated time to stdout 2021-11-23 15:01:44 -08:00
Evan Tschannen 6d9f134bf0 fix: merge needs to update on empty versions to release other mutations blocked by the empty version 2021-11-23 15:01:44 -08:00
Mohamed Oulmahdi fe98f1ea56 Add missing bind header for Windows (boost/bind/bind.hpp) 2021-11-22 15:50:32 +01:00
Renxuan Wang 22e34bd6b9 Replace <boost/bind.hpp> with <boost/bind/bind.hpp>.
This eliminates many useless warnings when compiling.
`#pragma message: The practice of declaring the Bind placeholders (_1, _2, ...) in the global namespace is deprecated. Please use <boost/bind/bind.hpp> + using namespace boost::placeholders, or define BOOST_BIND_GLOBAL_PLACEHOLDERS to retain the current behavior.`
2021-11-18 14:00:13 -08:00
Steve Atherton 035e0d6e52
Merge branch 'master' into bit-flipping-workload 2021-11-16 14:42:22 -08:00
Renxuan Wang 4630b0ccea Move DNS mock from SimExternalConnection to Sim2.
This is a revise PR of #5934. In simulation, we don't have direct access to SimExternalConnection.
2021-11-15 17:02:51 -08:00
Steve Atherton 508429f30d
Redwood chunked file growth and low priority IO starvation prevention (#5936)
* Redwood files now growth in large page chunks controlled by a knob to reduce truncate() calls for expansion.   PriorityMultiLock has limit on consecutive same-priority lock release.  Increased Redwood max priority level to 3 for more separation at higher BTree levels.

* Simulation fix, don't mark certain IO timeout errors as injected unless the simulated process has been set to have an unreliable disk.

* Pager writes now truncate gradually upward, one chunk at a time, in response to writes, which wait on only the necessary truncate operations.   Increased buggified chunk size because truncate can be very slow in simulation.

* In simulation, ioTimeoutError() and ioDegradedOrTimeoutError() will wait until at least the target timeout interval past the point when simulation is sped up.

* PriorityMultiLock::toString() prints more info and is now public.

* Added queued time to PriorityMultiLock.

* Bug fix to handle when speedUpSimulation changes later than the configured time.

* Refactored mutation application in leaf nodes to do fewer comparisons and do in place value updates if the new value is the same size as the old value.

* Renamed updatingInPlace to updatingDeltaTree for clarity.  Inlined switchToLinearMerge() since it is only used in one place.

* Updated extendToCover to be more clear by passing in the old extension future as a parameter.  Fixed initialization warning.
2021-11-12 13:47:07 -08:00
Steve Atherton e752fdd69f
Optimization in FlowTransport's use of XXHash to avoid a malloc() when using the stream API and to not use the stream API when a message being sent is contiguous within a single packet buffer block. (#5970) 2021-11-12 11:45:21 -08:00
Renxuan Wang d9965e0e5a Add DNS mock in SimExternalConnection.
This is a substep in supporting hostnames in cluster files. So that in simulation, we can add <hostname, ip addresses> mappings to mock DNS.
2021-11-11 17:05:05 -08:00
Markus Pilman a246c1555d
Merge pull request #5931 from sfc-gh-mpilman/features/all-unit-tests-in-ci
Make sure unit tests are run often enough
2021-11-10 08:41:04 -07:00
Tao Lin fdb3b72e35 Introduce GetRangeAndFlatMap to push computations down to FDB
Re-introduce #5609
2021-11-09 13:52:28 -08:00
negoyal 8fac798ece Fix a memory bug. 2021-11-09 10:32:17 -08:00
Markus Pilman 7df059570a Make sure unit tests are run often enough 2021-11-08 15:43:32 -07:00
sfc-gh-tclinkenbeard e4ca7e9511 Fix comment for RequestStream::tryGetReply 2021-11-04 12:59:50 -07:00
Renxuan Wang f15ceb5489 Add Hostname struct, and fromHostname in NetworkAddress struct. 2021-11-04 11:42:28 -07:00
Tao Lin 586cc3b102
Revert "Introduce GetRangeAndFlatMap to push computations down to FDB" 2021-11-04 08:46:56 -07:00
Tao Lin 6c98e35893 Rename Hop to FlatMap 2021-11-03 13:32:01 -07:00
Tao Lin 0853661d13 Introduce getRangeAndHop to push computations down to FDB 2021-11-03 13:21:16 -07:00
Steve Atherton dc3f46c2ae Change asserts to throw internal_error as these function calls should never be compiled out. 2021-11-03 11:00:12 -07:00
Steve Atherton f3d62ce002 Assert success on XXHash stream functions. 2021-11-02 23:40:53 -07:00
Steve Atherton 313a3dcd63 Change FlowTransport to use 64 bit XXHash instead of CRC32. 2021-11-02 23:16:03 -07:00
sfc-gh-tclinkenbeard d0c9cf4fb0 Enable mismatched-tags clang warning 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard ebcc023b6f Enable missing-field-initializers clang warning 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard 168e75bb1e Remove unused shouldNotHaveFriends* functions 2021-11-01 14:18:31 -07:00
negoyal 1e7338b6c3 Merge branch 'master' into bit-flipping-workload 2021-10-28 14:24:49 -07:00
negoyal 88e66533ad devFormat 2021-10-28 11:13:12 -07:00
Evan Tschannen 2208b04174
Merge pull request #5855 from sfc-gh-etschannen/blob_full_clean
Blob Granules V0
2021-10-26 09:57:35 -07:00
sfc-gh-tclinkenbeard 49a667c29b Improve const-correctness of INetwork 2021-10-25 14:42:31 -07:00
Josh Slocum 0ff8ddc2b6 Merge branch 'master' into blob_full_clean 2021-10-25 13:38:48 -05:00
Evan Tschannen 452404f0ee fix: disconnectTriggers cannot be yielded because we could send a reply to a replyPromiseStream while waiting on the delay 2021-10-24 19:18:03 -07:00
Josh Slocum 773886515e Merge branch 'feature-range-feed' into blob_full_clean 2021-10-22 11:07:51 -05:00
Evan Tschannen f1158371a7 Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
# Conflicts:
#	flow/error_definitions.h
2021-10-21 00:55:12 -07:00
Evan Tschannen e3cd6f8b6f fix: do not throw errors from sendError 2021-10-20 22:05:00 -07:00
Evan Tschannen 3f7df58a77 fixed a number of issues 2021-10-19 13:56:52 -07:00
Josh Slocum 912ef76f1c cleanup before merge 2021-10-18 17:11:14 -05:00
Trevor Clinkenbeard 504d0b71b2
Fix invalid memory access when dataDistribution actor is cancelled (#5791)
* Fix valgrind error when dataDistribution actor is cancelled

* Trace Sev30 when dataDistribution actor is cancelled outside of simulation

* Rethrow actor_cancelled error in dataDistribution catch block
2021-10-18 14:21:29 -07:00
Suraj Gupta b15e4f6b4e Fix reference cycle between BWData and GranuleRangeMetadata. 2021-10-18 17:03:27 -04:00
Josh Slocum 5f0ec0612a Merge branch 'feature-range-feed' into blob_full 2021-10-13 15:44:35 -05:00
Suraj Gupta 266a5b06fa Fix infinite loop. 2021-10-13 14:40:26 -04:00
negoyal f913dfed97 Merge branch 'master' into bit-flipping-workload 2021-10-11 16:34:57 -07:00
Steve Atherton 14efc4c251 Bug fix which only affects simulation: dataCopy was created after a potential wait so the source memory may have already been released. 2021-10-11 08:23:15 -06:00
negoyal 8d1e97b329 Minor changes. 2021-10-04 22:43:48 -07:00
Suraj Gupta 282f9d35cd Cleanup comments and debugging code. 2021-10-04 11:07:08 -04:00
Suraj Gupta 4d54669ccd Recruit the blob workers via blob manager.
In this PR, the blob manager now recruits blob workers
(via communication with the cluster controller). Blob workers
are onboarded as blob worker processes enter the cluster.
2021-10-04 11:07:08 -04:00
Chang Liu 8427e40cbe Fix roll trace event issue
Description

Testing
2021-09-24 09:53:32 -07:00
Chang Liu 48990058a3 Fix roll trace event issue
Description

Testing
2021-09-24 09:53:32 -07:00