Convert tests/TestRunner/ into a proper Python package with standard
setuptools packaging, eliminating all sys.path manipulation and
PYTHONPATH usage.
Verified in Docker (rockylinux9): ctest -R test_venv_setup passes 2/2
Add dependency on fdb_python_options target in prepare_binding_test_files
to ensure fdboptions.py and apiversion.py are generated before the copy
step tries to use them.
* Add --reseed-time option to reset random seed during simulation
Adds a new command line option --reseed-time that allows resetting the
deterministic random seed at a random time during simulation. When
provided with a time value (e.g., 100), the simulator will reset the
random seed to a truly random value at a random time between [0, time].
This feature enables testing different execution paths from the same
initial simulation state while maintaining determinism up to the reset
point.
Changes:
- Add resetSeed() method to IRandom interface with default no-op implementation
- Implement resetSeed() in DeterministicRandom class to reinitialize RNG
- Add --reseed-time command line option to fdbserver
- Add reseedTime parameter to simulationSetupAndRun()
- Add reseedRandomAtTime() actor to schedule seed reset
- Use platform::getRandomSeed() for truly random new seed value
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* TestHarness2: Add support for running tests from argument file
This commit adds the ability to run specific tests with predefined
parameters by passing an argument file to TestHarness2. This is useful
for reproducing specific test failures or running tests with exact
configurations.
Changes:
- Added --test-args-file option to config.py
- Implemented parse_test_args_file() to parse argument files containing
test parameters in the format:
'-f fast/CycleTest.toml -s 315315 -b off --reseed-time 100'
- Modified TestRun to accept and forward extra_args to fdbserver
- Modified TestRunner.run_tests() to:
* Accept extra_args and override_buggify parameters
* Skip test statistics tracking when running from args file
* Forward all unrecognized arguments to fdbserver command line
- Added example_test_args.txt demonstrating the format
The track_stats parameter prevents assertion failures when the specified
test is not in the test_picker's dictionary (which happens when running
a specific test that may not match the configured filters).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Joshua: Support test_args.txt file for reproducing specific test runs
Modified correctnessTest.sh to automatically detect and use test_args.txt
file if present in the working directory. This enables easy reproduction of
specific test failures in Joshua by simply including a test_args.txt file
with the test parameters.
When test_args.txt exists, the script passes --test-args-file option to
TestHarness2, which will run the specific test with the exact parameters
specified in the file instead of randomly selecting a test.
Example test_args.txt content:
-f tests/fast/CycleTest.toml -s 315315 -b off --reseed-time 100
This allows developers to reproduce simulation test failures by copying
the exact test parameters from Joshua logs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* CMake: Optionally include test_args.txt in correctness package
Modified create_correctness_package() to automatically include test_args.txt
in the correctness tarball if it exists in the source directory at CMake
configuration time.
This allows developers to create reproducible test packages by:
1. Creating test_args.txt in the source root with specific test parameters
(e.g., '-f tests/fast/CycleTest.toml -s 315315 -b off --reseed-time 100')
2. Running cmake to reconfigure
3. Running ninja package_tests to build the package
When the package is extracted and run in Joshua, correctnessTest.sh will
automatically detect test_args.txt and run the specific test instead of
randomly selecting one.
If test_args.txt doesn't exist, the package is built normally without it,
maintaining backward compatibility.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* TestHarness2: Disable determinism check when using test_args_file
Modified run_tests() to skip the unseed mismatch (determinism) check when
running tests from test_args.txt file. The determinism check now requires
track_stats=True, which means it only runs during normal random test
selection, not when running specific tests via --test-args-file.
Rationale:
- When reproducing a specific test failure via test_args.txt, users want to
run that exact test configuration once, not have it randomly run twice
- The determinism check adds overhead and complexity that isn't needed for
reproduction scenarios
- The check uses config.random which would make reproduction less predictable
This change makes test_args.txt behavior more focused and predictable for
debugging and reproduction workflows.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* TestHarness2: Include ResettingRandomSeed events in test output
Added a handler for ResettingRandomSeed trace events to include them in
the test summary output. This makes random seed reset operations visible
in Joshua logs and test results.
The handler captures:
- NewSeed: The new random seed value
- Time: Simulation time when the reset occurred
This is particularly useful when debugging tests that use --reseed-time
option, as it provides visibility into when and how the random seed was
changed during the test run.
Example output:
<ResettingRandomSeed NewSeed="67890" Time="100.5"/>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Use non-deterministic random for reseed time
To truly randomize the seed reset time.
* clang-format fix
* Address review comments
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* Make FDB_USE_CSHARP_TOOLS authoritative across the build
Historically, FDB_USE_CSHARP_TOOLS acted as a preference hint, and parts of the build could still probe for or assume the presence of C# tooling even when it was disabled.
This change makes the option authoritative and consistently honored across the build system. C# tooling is now used only when explicitly enabled and available, and all downstream assumptions are gated accordingly.
The default configuration and tool preference order remain unchanged.
* cmake files changes
* WIP: tmp test
* Honor reviewer feedback on C# toolchain detection and actor comparison
- Stop assuming C# tooling availability on Windows; explicitly probe for
.NET using find_program.
- Prefer .NET over mono on all platforms, with mono used only as a fallback.
- Fail explicitly when FDB_USE_CSHARP_TOOLS=ON but no C# toolchain is found.
- Preserve Python/C# actor output comparison when C# tooling is available,
skipping it only when C# is explicitly disabled or unavailable.
- Simplify Python argument parsing and remove unnecessary textwrap usage.
These features have been previously marked for deletion per PR #12400.
This change necessarily affects a lot of files. In general I found it preferable to cut along the FDB <-> tenant boundary, rather than try to cut tenant into multiple pieces, stitch the Frankenstein tenant implementation back together with FDB, and generally remove the limbs one by one. So it is a single big deletion.
Note that some tenant-related metadata has been written in a non-flag-controlled manner by prior releases and probably must be ignored indefinitely. Fortunately this is isolated to include/fdbclient/ClientLogEvents.h. (Details: deleting an Optional from a serialized struct results in deserialization of garbage in upgrade tests. The serialized nullopt to indicate "no Tenant" is formally part of FDB persistent metadata even in FDB clusters that never would have enabled the tenant feature.)
During the course of testing these changes, many interesting bugs were encountered. I won't discuss details of them here. Causes range from flat out damage (by me) to production code in the course of removing tenant related bits (mainly in NativeAPI.actor.cpp and CommitProxy.actor.cpp), damage to various workload files (particularly FuzzApiCorrectness.actor.cpp, which is very sensitive to changes), and many toml files needing updated test flags/options.
More testing details: https://quip-apple.com/Zr6VAycxoli9
20251209-012852-gglass-8ff850b772d868f2 compressed=True data_size=35311687 duration=21671404 ended=500000 fail_fast=1000 max_runs=500000 pass=500000 priority=100 remaining=0 runtime=2:31:30 sanity=False started=500000 stopped=20251209-040022 submitted=20251209-012852 timeout=5400 username=gglass
* remove some unneeded tests, and remove mentions of deleted tests from tests/CmakeLists.txt
* Initiate removal of metacluster. NOTE: this seems to also want removal of tenant. Consider removing them together.
* work on removing metacluster
* delete files with `Tenant` in the name, having reviewed them to ensure that they basically contain what the name implies
* fdb_c.h: remove prototypes for C API methods which have been deleted (blob granule) or which are so long deprecated that they are outside any reasonable/documented support window
* Surgical removal of tenant references from files in bindings/ top level directory. Compilation not yet attempted.
* Surgical removal of tenant related stuff from fdbcli/ top level directory. Compilation not yet attempted.
* Misc tenant code removal, and other stuff which I think may not be needed. Compilation still not attempted.
* Remove more tenant or tenant-adjacent or blob-granule-adjacent stuff. Or at least stuff that looks adjacent to that stuff. Not compiled or tested.
* Start removing Tenant stuff from fdbclient/. Far from complete. Compilation not attempted.
* Remove tenant references from many source files. There are still about 7 principal fdbclient/ and fdbserver/ files with a lot of tenant logic left to delete. Also, all of fdbserver/workloads needs to be looked at. Still have not attempted compilation.
* Remove tenant entanglement from watch functionality
* Remove tenant stuff from fdbserver/tester.actor.cpp
* Delete metacluster workloads
* Remove tenant related stuff from workloads. Also taken the liberty of removing some functionality that appears unused or untestable by Apple.
* Checkpoint tenant removal from FuzzApiCorrectness.actor.cpp
* NativeAPI.actor.cpp: `Tenant` has left the building.
* SimulatedCluster.actor.cpp: `Tenant` has left the building
* DDShardTracker.actor.cpp: Tenant evicted
* storageserver.actor.cpp: `tenant` has left the building.
* fdbserver/workloads/FuzzApiCorrectness.actor.cpp: remove tenant references, but some lingering cleanup needed in `loadAndRun`
* FileBackupAgent.actor.cpp: tenant has left the building
* CommitProxyServer.actor.cpp: remove tenant
* Remove more tenant references from misc files such as bindings tests, documentation, and some fdbserver headers I left earlier
* Fix missing-file errors in CMakeLists.txt files. This is the first attempt to compile this stuff.
* checkpoint misc changes to fix compile errors
* checkpoint more compile fixes
* StorageServerInterface.h: put back more verify() calls
* More misc compile fixes
* whole bunch of misc fixups including some code put-backs to address compile errors
* More compile fixes
* More compile fixes. Still does not compile.
* incremental compile fixing
* ...
* ...
* Checkpoint a bunch of compile fixes. Not quite there but getting closer
* More compile fixes. There seem to be about 10 files left, mainly CommitProxyServer.actor.cpp and storageserver.actor.cpp
* IT COMPILES NOW. THIS IS STILL ALL UNTESTED. Unsurprisingly, CommitProxyServer.actor.cpp and storageserver.actor.cpp took the most tweaking.
The updates in CMakeLists.txt and workloads/UnitTests.actor.cpp are basically trivial and mainly reflect
the ordering of dependencies -- that stuff didn't get attempted until all of fdbserver compiled.
* Put back one block relating to encryption at rest mode. Simplify some TODO(gglass) instances.
* Put back some encryption related knobs
* remove `enable_tenants` from local_cluster.py to maybe fix some ctests
* Remove tenant related options from toml files.
* feature-status.md: add a line for encryption at rest, which seems to have been added for multi-tenant; status is now in doubt
* Fix a pretty bad bug introduced in tenant deletion; ensure we dont attempt to construct a std::string of negative length
* workloads/FuzzApiCorrectness.actor.cpp: avoid division by zero
* flow/Platform.actor.cpp: add a try/catch wrapper around side threads; emit a better addr2line type command
* NativeAPI.actor.cpp: fix a bug introduced in tenant removal relating to reporting conflicting keys under conflictingKeysRange
* ReportConflictingKeys.actor.cpp: separate an ANDed assert into two asserts
* SpecialKeySPaceCorrectness.actor.cpp: put back some logic removed with tenant removal. This test was failing due to a bug with conflict key range reporting. Fixed separately in NativeAPI.actor.cpp.
* remove QuotaCommand.actor.cpp
* Force disable tenant and encryption on disk in upgrade tests
* Add back file I guess I deleted? who knows
* put back another file
* design/feature-status.md: update the new row for encryption at rest to firm up the claim that it is experimental, unowned, and scheduled for deletion
* Remove EncryptKeyProxyTest since we do not use it
* new file tests/slow/BulkDumpingS3WithChaos.toml: remove tenantModes setting
* Undo damage to pushToBackupMutations() from removing tenant feature. This caused inverted_range errors and failed commits in backup related simulations.
* tests/restarting/from_7.4.0/Snap*-1: ensure that tenantModes = disabled
* Try again on workloads/FuzzApiCorrectness.actor.cpp
* simplify tenant-free (mostly) FuzzApiCorrectness workload code
* try harder to remove lingering tenant-related brokenness from FuzzApiCorrectness.actor.cpp
* Explicitly specify tenantModes = ['disabled'] in all the -1 restart files
* Remove tenantModes from 7.1-based upgrade tests as its an unknown option. Hopefully the code doesnt actually turn on tenant stuff
* do not specify tenantModes in downgrade tests
* Downgrade test to_7.4.5: dont say tenantModes
* more tenantModes updates
* Remove a legacy allowDefaultTenant that no longer is meaningful in downgrade to 8.0
* Put back empty Optional<TenantName> turdlets into serialized log events to avoid breaking ClientTransactionProfilingCorrectness upgrade tests (even with tenantMode = disabled)
* disable encryption on a few more upgrade related test cases. That feature is slated for removal anyway
* Remove unneeded workload files that have been subject to #if 0 for a while. Remove commented out block in ClusterRecovery
* disable encryption in more upgrade tests
* Remove choice four-letter words from commentary
* Format 42 files
* Try to fix a doc bug failing the CI build
* More doc compilation error fixes
* Delete more tenant junk from documentation
* fix spelling mistake in comment
* Remove deleted cross-references from documentation. This necessitated editing release 3.0.0 release notes, which is insane.
* Remove more tenant stuff from bindings tests
* Remove more tenant bits from design/ files
* Remove more tenant related stuff
* Delete more tenant references. Put back ten-ant spellings as tenant now that grep output is substantially reduced.
* Put back some tenant stuff into apitester; its deletion seems to have introduced bugs. Also whine about comments some more, because, really, the comments deserve it.
* Updates to workload files and one other thing based on review comments
* de-actorify decodeKVPairs
* format one source file
* Restore transaction tagging doc
* Restore throttle doc details in administration.rst
* Restore fdbserver/workloads/GetEstimatedRangeSize.actor.cpp and associated toml file, minus tenant stuff
* bindings/c/test/{shim related}: update comments and disable functionality that no longer works post-tenant
* put the cli-throttle tag back in
* bindingtester: fix python syntax errors
* remove useless comment
* Remove comment about useless comments, and remove the useless comments
This is the first experimental feature to be deleted in the list published at PR #12400.
There is more code here than I anticipated. It is about 40,000 lines total, of which about three quarters are in dedicated files which I am deleting, and about one quarter is in shared files. That means about 10k lines in shared files, which is the stuff we tend to notice day to day (that plus the test failures on heretofore not-yet-disabled test cases, which I am now deleting).
I ran 3 million simulations, mostly against 692df86 or very similar code (differing by one TraceEvent). This was prior to syncing with upstream/main, which had no conflicts and from which I don't expect problems. The number of failures in these runs was about 8. We looked at them and believe there is a high likelihood that these are existing issues not related to the changes in this PR. More details on these failures can be found in docs linked from here: https://quip-apple.com/MN7gAyXLjgyn
* change Long Term status for unowned features for "scheduled for deletion" where applicable
* Relax wording about scheduled for deletion features
* Delete blob granule feature. WIP. Does not compile.
* more incremental hacking to remove / comment out blob granule related code
* more hacking to remove blob granule related code, e.g. blob manager and blob migrator roles
* delete more blob granule stuff
* more hacking
* more hacking
* more hacking
* More changes to remove blob granule related code. IT COMPILES NOW
* dont try to run AuthzSecurity tests as we have deleted that workload as part of this effort
* delete more stuff that matches, abbreviates, or smells like blob granule related
* EncryptKeyProxy: dont do blobMetadata stuff, because that is not used and support is being removed
* delete more references to blob granule stuff
* SimulationConfig::setEncryptionAtRestMode: always use DISABLED; also disable EncryptKeyProxyTest.toml
* format code
* manual update to bindings/java/src/tests.cmake to remove a deleted file
* fix compile errors. I guess by default I dont build Java bindings
* remove unneeded blob granule functions rather than #if..#endif them out
* remove more code in #if..#endif
* remove more code in #if 0..#endif
* revert changes to fdb_c.h in preparation for marking removed API calls as removed
* rework C API declarations to in preparation for marking blob granule APIs as removed
* deprecate removed glob granule related API functions as of version 740 (and add a comment to request a justification of this convention)
* make progress on broken ctests. E.g. 1) python does not need to do blob granule stuff. 2) authz tests seemingly not needed
* remove blob granule stuff from Java and Python APIs and fix test runner stuff so that ctests pass
* reformat comments to fix compile error. FIXME: why is this error not happening on the default compile commands we use
* hacks all the way down to try to fix the Mac build
* add pointed comment about the perceived pointlessness of the API deprecation scheme embodied in this source file
* really serious about the C++ style comments, arent we
* remove commented-out code from prior iterative efforts
* put back undeleted code in original order
* delete commented-out code
* update feature-status.md to say blob granule is mostly deleted
* upgrade `mostly deleted` to `has been deleted`
* [testing] Automatically discover unit-test and register as ctest
This patch adds `collect_unit_tests()` to CMake which searches over
the codebase and finds all the unit-tests written using Flow's TEST_CASE
macro and register as ctest.
The test then can be then run using ctest command or directly via Test
Explorer in VSCode.
* Update CMakeLists.txt
* Check failed tests
* Update TestDirectory.py to create more unique directory
* Put the feature behind flag
* [fdbserver] workaround the FRT type layout issue to get Swfit getVersion working
* MasterData.actor.h: fix comment typo
* masterserver.swift: some tweaks
* masterserver.swift: remove getVersion function, use the method
* masterserver.swift: print replied version to output for tracing
* [swift] add radar links for C++ interop issues found in getVersion bringup
* Update fdbserver.actor.cpp
* Migrate MasterData closer to full reference type
This removes the workaround for the FRT type layout issue, and gets us closer to making MasterData a full reference type
* [interop] require a new toolchain (>= Oct 19th) to build
* [Swift] fix computation of toAdd for getVersion Swift implementation
* add Swift to FDBClient and add async `atLeast` to NotifiedVersion
* fix
* use new atLeast API in master server
* =build fixup link dependencies in swift fdbclient
* clocks
* +clock implement Clock using Flow's notion of time
* [interop] workaround the immortal retain/release issue
* [swift] add script to get latest centos toolchain
* always install swift hooks; not only in "test" mode
* simulator - first thing running WIP
* cleanups
* more cleanup
* working snapshot
* remove sim debug printlns
* added convenience for whenAtLeast
* try Alex's workaround
* annotate nonnull
* cleanup clock a little bit
* fix missing impls after rebase
* Undo the swift_lookup_Map_UID_CommitProxyVersionReplies workaround
No longer needed - the issue was retain/release
* [flow][swift] add Swift version of BUGGIFY
* [swiftication] add CounterValue type to provide value semantics for Counter types on the Swift side
* remove extraneous requestingProxyUID local
* masterserver: initial Swift state prototype
* [interop] make the Swiftied getVersion work
* masterserver - remove the C++ implementation (it can't be supported as state is now missing)
* Remove unnecessary SWIFT_CXX_REF_IMMORTAL annotations from Flow types
* Remove C++ implementation of CommitProxyVersionReplies - it's in Swift now
* [swift interop] remove more SWIFT_CXX_REF_IMMORTAL
* [swift interop] add SWIFT_CXX_IMMORTAL_SINGLETON_TYPE annotation for semanticly meaningful immortal uses
* rename SWIFT_CXX_REF_IMMORTAL -> UNSAFE_SWIFT_CXX_IMMORTAL_REF
* Move master server waitForPrev to swift
* =build fix linking swift in all modules
* =build single link option
* =cmake avoid manual math, just get "last" element from list
* implement Streams support (#18)
* [interop] update to new toolchain #6
* [interop] remove C++ vtable linking workarounds
* [interop] make MasterData proper reference counted SWIFT_CXX_REF_MASTERDATA
* [interop] use Swift array to pass UIDs to registerLastCommitProxyVersionReplies
* [interop] expose MasterServer actor to C++ without wrapper struct
* [interop] we no longer need expose on methods 🥳
* [interop] initial prototype of storing CheckedContinuation on the C++ side
* Example of invoking a synchronous swift function from a C++ unit test. (#21)
* move all "tests" we have in Swift, and priority support into real modules (#24)
* Make set continuation functions inline
* Split flow_swift into flow_swift and flow_swift_future to break circular dependency
* rename SwiftContinuationCallbackStruct to FlowCallbackForSwiftContinuation
* Future interop: use a method in a class template for continuation set call
* Revert "Merge pull request #22 from FoundationDB/cpp-continuation" (#30)
* Basic Swift Guide (#29)
Co-authored-by: Alex Lorenz <arphaman@gmail.com>
* Revert "Revert "Merge pull request #22 from FoundationDB/cpp-continuation" (#30)"
This reverts commit c025fe6258.
* Restore the C++ continuation, but it seems waitValue is broken for CInt somehow now
* disable broken tests - waitValue not accessible
* Streams can be async iterated over (#27)
Co-authored-by: Alex Lorenz <arphaman@gmail.com>
* remove work in progress things (#35)
* remove some not used (yet) code
* remove expose func for CInt, it's a primitive so we always have witness info (#37)
* +masterdata implement provideVersions in Swift (#36)
* serveLiveCommittedVersion in Swift (#38)
* Port updateLiveCommittedVersion to swift (#33)
Co-authored-by: Konrad `ktoso` Malawski <konrad_malawski@apple.com>
* Implement updateRecoveryData in Swift (#39)
Co-authored-by: Alex Lorenz <arphaman@gmail.com>
* Simplify flow_swift to avoid multiple targets and generate separate CheckedContinuation header
* Uncomment test which was blocked on extensions not being picked up (#31)
* [interop] Use a separate target for Swift-to-C++ header generation
* reduce boilerplate in future and stream support (#41)
* [interop] require interop v8 - that will fix linker issue (https://github.com/apple/swift/issues/62448)
* [interop] fix swift_stream_support.h Swift include
* [interop] bump up requirement to version 9
* [interop] Generalize the Flow.Optional -> Swift.Optional conversion using generics
* [WIP] masterServer func in Swift (#45)
* [interop] Try conforms_to with a SWIFT_CONFORMS_TO macro for Optional conformance (#49)
* [interop] include FlowOptionalProtocol source file when generating Flow_CheckedContinuation.h
This header generation step depends on the import of the C++ Flow module, which requires the presence of FlowOptionalProtocol
* conform Future to FlowFutureOps
* some notes
* move to value() so we can use discardable result for Flow.Void
* make calling into Swift async funcs nicer by returning Flow Futures
* [interop] hide initial use of FlowCheckedContinuation in flow.h to break dependency cycle
* [fdbserver] fix an EncryptionOpsUtils.h modularization issue (showed up with modularized libc++)
* Pass GCC toolchain using CMAKE_Swift_COMPILE_EXTERNAL_TOOLCHAIN to Swift's clang importer
* [interop] drop the no longer needed libstdc++ include directories
* [cmake] add a configuration check to ensure Swift can import C++ standard library
* [swift] include msgpack from msgpack_DIR
* [interop] make sure the FDB module maps have 'export' directive
* add import 'flow_swift' to swift_fdbserver_cxx_swift_value_conformance.swift
This is needed for CONFORMS_TO to work in imported modules
* make sure the Swift -> C++ manually bridged function signature matches generated signature
* [interop][workaround] force back use of @expose attribute before _Concurrency issue is fixed
* [interop] make getResolutionBalancer return a pointer to allow Swift to use it
We should revert back to a reference once compiler allows references again
* [interop] add a workaround for 'pop' being marked as unsafe in Swift
* masterserver.swift: MasterData returns the Swift actor pointer in an unsafe manner
* Add a 'getCopy' method to AsyncVar to make it more Swift friendly
* [interop] bump up the toolchain requirement
* Revert "[interop][workaround] force back use of @expose attribute before _Concurrency issue is fixed"
This reverts commit b01b271a76.
* [interop] add FIXME comments highlighting new issue workarounds
* [interop] adopt the new C++ interoperability compiler flag
* [interop] generate swift compile commands
* Do not deduplicate Swift compilation commands
* [interop] generate swift compile commands
* Do not deduplicate Swift compilation commands
* flow actorcompiler.h: add a SWIFT_ACTOR empty macro definition
This is needed to make the actor files parsable by clangd
* [cmake] add missing dependencies
* experimental cross compile
* [cmake] fix triple in cross-compiled cmake flags
* [interop] update to interop toolchain version 16
* [x-compile] add flags for cross-compiling boost
* cleanup x-compile cmake changes
* [cmake] fix typo in CMAKE_Swift_COMPILER_EXTERNAL_TOOLCHAIN config variable
* [interop] pass MasterDataActor from Swift to C++ and back to Swift
* [fdbserver] Swift->C++ header generation for FDBServer should use same module cache path
* Update swift_get_latest_toolchain.sh to fetch 5.9 toochains
* set HAVE_FLAG_SEARCH_PATHS_FIRST for cross compilation
* Resolve conflicts in net2/sim2/actors, can't build yet
* undo SWIFT_ACTOR changes, not necessary for merge
* guard c++ compiler flags with is_cxx_compile
* Update flow/actorcompiler/ActorParser.cs
Co-authored-by: Evan Wilde <etceterawilde@gmail.com>
* update the boost dependency
* Include boost directory from the container for Swift
* conform flow's Optional to FlowOptionalProtocol again
* Guard entire RocksDBLogForwarder.h with SSD_ROCKSDB_EXPERIMENTAL to avoid failing on missing rocksdb APIs
* remove extraneous merge marker
* [swift] update swift_test_streams.swifto to use vars in more places
* Add header guard to flow/include/flow/ThreadSafeQueue.h to fix moduralization issue
* Update net and sim impls
* [cmake] use prebuilt libc++ boost only when we're actually using libc++
* [fdbserver] Swift->C++ header generation for FDBServer should use same module cache path
* fixups after merge
* remove CustomStringConvertible conformance that would not be used
* remove self-caused deprecation warnings in future_support
* handle newly added task priority
* reformatting
* future: make value() not mutating
* remove FIXME, not needed anymore
* future: clarify why as functions
* Support TraceEvent in Swift
* Enable TraceEvent using a class wrapper in Swift
* prearing WITH_SWIFT flag
* wip disabled failing Go stuff
* cleanup WITH_SWIFT_FLAG and reenable Go
* wip disabled failing Go stuff
* move setting flag before printing it
* Add SWIFT_IDE_SETUP and cleanup guides and build a bit
* Revert "Wipe packet buffers that held serialized WipedString (#10018)"
This reverts commit e2df6e3302.
* [Swift] Compile workaround in KeyBackedRangeMap; default init is incorrect
* [interop] do not add FlowFutureOps conformance when building flow clang module for Flow checked continuation header pre-generation
* make sure to show -DUSE_LIBCXX=OFF in readme
* readme updates
* do not print to stderr
* Update Swift and C++ code to build with latest Swift 5.9 toolchain now that we no longer support universal references and bridge the methods that take in a constant reference template parameter correctly
* Fix SERVER_KNOBS and enable use them for masterserver
* Bump to C++20, Swift is now able to handle it as well
* Put waitForPrev behind FLOW_WITH_SWIFT knob
* Forward declare updateLiveCommittedVersion
* Remove unused code
* fix wrong condition set for updateLiveCommittedVersion
* Revert "Revert "Wipe packet buffers that held serialized WipedString (#10018)""
This reverts commit 5ad8dce052.
* Enable go-bindings in cmake
* Revert "Revert "Wipe packet buffers that held serialized WipedString (#10018)""
This reverts commit 5ad8dce052.
* USE_SWIFT flag so we "build without swift" until ready to by default
* uncomment a few tests which were disabled during USE_SWIFT enablement
* the option is WITH_SWIFT, not USE
* formatting
* Fix masterserver compile error
* Fix some build errors.
How did it not merge cleanly? :/
* remove initializer list from constructor
* Expect Swift toolchain only if WITH_SWIFT is enabled
* Don't require Flow_CheckedContinuation when Swift is disabled
* Don't compile FlowCheckedContinuation when WITH_SWIFT=OFF
* No-op Swift macros
* More compile guards
* fix typo
* Run clang-format
* Guard swift/bridging include in fdbrpc
* Remove printf to pass the test
* Remove some more printf to avoid potential issues
TODO: Need to be TraceEvents instead
* Remove __has_feature(nullability) as its only used in Swift
* Don't use __FILENAME__
* Don't call generate_module_map outside WITH_SWIFT
* Add some more cmake stuff under WITH_SWIFT guard
* Some more guards
* Bring back TLSTest.cpp
* clang-format
* fix comment formatting
* Remove unused command line arg
* fix cmake formatting in some files
* Address some review comments
* fix clang-format error
---------
Co-authored-by: Alex Lorenz <arphaman@gmail.com>
Co-authored-by: Russell Sears <russell_sears@apple.com>
Co-authored-by: Evan Wilde <etceterawilde@gmail.com>
Co-authored-by: Alex Lorenz <aleksei_lorenz@apple.com>
Co-authored-by: Vishesh Yadav <vishesh_yadav@apple.com>
Co-authored-by: Vishesh Yadav <vishesh3y@gmail.com>
A script, rocksdb_logtool.py, is available to upload/download generated
XML/JSON log files when test harness 2 detects that the test is failed,
and the script detects that the test is using RocksDB.
To upload, no actions is needed, using the regular
```
joshua start --tarball correctness.tgz
```
The build system will automatically pack the rocksdb_logtool.py into the
tarball and test harness 2 will call the script if it thinks the test is
failed.
To download, simply provide the ensemble id and test uid, e.g.
```
python3 rocksdb_logtool.py download --ensemble-id
20230222-204240-xiaogesu-cb6ea277a898f134 --test-uid
ab6fb792-088f-49d6-92d2-43bc4fb81668
```
Note the test UID can be retrieved by
```
joshua tail ensemble_id
```
output, it is in the field `TestUID` in <Test> element of test harness 2
generated XML.
For convenience, it is possible to do a
```
python3 rocksdb_logtool.py list --ensemble-id ensemble-id
```
to generate all possible download commands for failed tests. However,
the list subcommand will *NOT* verify if the test failure is coming from
RocksDB, i.e. other test failues may be included and it is the caller's
responsibility to verify. If the test is not RocksDB related, the
download would fail as nothing is uploaded.
* get_client_status: add to Java bindings
* Add get_client_status in python binding
* Make python unit tests available as ctest
* Fix file name in the copyright header
* Fix library path name for mac
* get_client_status: Minor changes in Java & Python bindings addressing review comments
* Rename fdb_python_unit_tests to unit_tests
bindingtester2-${FDB_VERSION} is generated in packages/ directory.
It is possible to run the test by
j start --tarball bindingtester2-7.3.0.tar.gz --max-runs 1000
and submit the test to joshua system. Joshua will run the tests in
parallel and tests will not impact each other. Yet it will take long
time for each run.
Add atomic op functions to the tenant object in Python.
Fix a Python tester bug for tenant operations.
Fix a build error with the binding tester package.
* Auto-formatting fdb_c_shim_tests.py
* Define the old release versions used for testing in one place; Using PYTHONPATH to find python modules from other project directories
* Remove obsolete upgrade tests; add direct upgrade tests from older versions
* Trigger tests
* Add a cmake option ENABLE_LONG_RUNNING_TESTS to generate separate correctness package for long running tests
* Remove the unnessary long_running_tests directory
* Use the default bash files
* Run go unit tests in ctest
* Remove unnecessary cgo directives
* go: Allow access to TransactionOptions from ReadTransaction
Closes#6265
* Try go get instead of go install
My understanding is that this should download dependencies (e.g. xerrors)
* Add -d flag to match go command for building libraries
* Avoid concurrent calls to `go get`
Rodrigo Munoz