* rename task-handle mutex
* rename TimerClosure
* tmp commit, building, not tested
* add test for client connection to a non-listening port
* fix posix EE tests
* re-fix windows test suite after posix compatibility
* (unfinished, backup): passing the suite's NonExistingListener client test
* remove redundant windows client test
* integrate IOCP worker loop
* initial commit of echo test tool; fixes
* move echo client to test_suite/tools; I do not yet like the setup, it's about time for a macro that generates all useful test/tool targets
* cleanup
* add --target arg to echo_client. requires URI
* Automated change: Fix sanity tests
* build fixes
* fix
* fix
* reviewer feedback
* warning fix
* delete logic on cancellation
* fix
* cancel connect deadlock; improved template code
* fix build failure with linux environments building windows targets
* fix
* fix
* no ++ for atomics
* remove the test changes, to be landed separately
* Automated change: Fix sanity tests
* remnants
Co-authored-by: drfloob <drfloob@users.noreply.github.com>
* Revert "Revert "xDS stateful session affinity: add config plumbing (#31827)" (#31873)"
This reverts commit 4f15d3dcf9.
* fix build for compilers too dumb to recognize the full set of enum values
* Add type annotation to _channel.py and _server.py
* Undo changes in init.py
* changes based on comments.
* format changes
* fix pytype errors
* Changes based on comments
* Fix circular import
* fix pytype and pylint
* Fix format issues
* Fix some cython type issues
* Change ManagedCallType to IntegratedCallFactory
* fix format
* undo some mistakes
Add a check that SEND_STATUS_FROM_SERVER and RECV_MESSAGE are not in the same batch.
This is necessary pre-work for #31204 and implements part of grpc/proposal#336.
Also eliminates fling instead of updating it:
My expectation is nobody has looked at this corner in many years
It's not a benchmark we want: concentrating on a microbenchmark that doesn't include a binding layer caused us to favor designs that emphasized a lightweight core at the expense of a expensive bindings. We should consider the whole.
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Actually record CPU usage
* Do the client path too
* Actually build the image from the PR
* Revert "Actually build the image from the PR"
This reverts commit 3afa5cf3c3.
* Support importing on Windows
* Declarative JSON parser
* Automated change: Fix sanity tests
* fix
* shrinking stuff a little
* static vtables
* separate fns
* simpler?
* make maps work
* windows fixes
* Automated change: Fix sanity tests
* simplify code
* Automated change: Fix sanity tests
* vtable-test
* dont always create vec/map impls for every type
* comments
* make error consistent
* move method private
* progress
* durations!
* Automated change: Fix sanity tests
* fix
* fix
* fix
* Automated change: Fix sanity tests
* post-load
* Automated change: Fix sanity tests
* document JsonPostLoad() and add static_assert
* don't copy field names, to avoid length limitations
* use absl::Status
* accept either string or number for numeric values
* add test for direct data member of another struct type
* remove unused method
* add support for retaining part of the JSON wirthout processing
* update test for changes in Json::Parse() API
* add absl::optional support
* Automated change: Fix sanity tests
* fix tests, improve error messages, and add overload to parse to existing object
* remove overload of LoadFromJson()
* change special case for Json to instead use Json::Object
* rename resolver_result_parsing to client_channel_service_config
* split up service_config_test into a separate test for each component
* convert client channel service config parser to new API
* fix build
* converted retry global params
* convert retry method-level parsing, but still need to find a way to control parsing via a channel arg
* improve error structure, add missing types, and improve tests
* clang-format
* Automated change: Fix sanity tests
* fix build
* add LoadJsonObjectField(), add LoadFromJson() overload that takes an ErrorList parameter, and add tests for parsing bare top-level types
* fix msan
* Automated change: Fix sanity tests
* fix error message
* Automated change: Fix sanity tests
* fix test
* add ability to disable fields
* plumb in channel args to disable parsing
* Automated change: Fix sanity tests
* use const char* instead of absl::string_view for enable_key
* fix resolver_component_test
* Automated change: Fix sanity tests
* work around mac build problem
* Automated change: Fix sanity tests
* work around gcc6 problem
* Automated change: Fix sanity tests
* fix build
* fix build
* don't use alternative builder in tests
* convert message size service config parser
* convert fault injection service config parser
* rename files
* add specialization for unique_ptr
* avoid moves in client channel service config parser
* avoid moves in retry service config parser, and do some cleanup
* avoid moves in message_size service config parser
* avoid moves for fault injection service config parser, and use internal channel arg
* convert rbac service config parser
* clang-format
* WIP
* convert top-level service config parser
* clang-format
* fix build
* fix rbac service config parser test
* fix signed-ness problem and reversed-conditional bug
* fix unused param
* fix json_string method
* fix max message length defaults
* add copy ctors to appease windows compiler
* fix RLS LB config parser test
* fix name resolution test
* fix build
* work around gmock portability bug
* fix sanity
* add missing build dep
* make RBAC principal and permissions movable, not copyable
* Revert "make RBAC principal and permissions movable, not copyable"
This reverts commit 53315bccc9.
* attempt to simplify HeaderMatcher and StringMatcher parsing
* more bloat reduction in RBAC service config parser
* fix sanity
* add missing build dep
* attempt work-around for MSVC bug
* Revert "attempt work-around for MSVC bug"
This reverts commit e54c89e1e4.
* attempt work-around for Windows build problem
* try another work-around
* fix sanity
* appease clang-tidy
* generate_projects
* attempt to fix Windows build
* more windows fixes
* more windows fix
* yet more windows fixes
* try without noexcept
* remove unnecessary boilerplate
* code review changes
* fix breakage
Co-authored-by: Craig Tiller <craig.tiller@gmail.com>
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Co-authored-by: Craig Tiller <ctiller@google.com>
Co-authored-by: markdroth <markdroth@users.noreply.github.com>
* XdsBootstrap: move two more methods out of the interface
* Automated change: Fix sanity tests
* XdsClient: add unit test
* Automated change: Fix sanity tests
* fix memory leaks
* add helper method
* add unsubscription
* add test for multiple subscriptions
* clang-format
* fix build
* fix flakiness
* add checking for other node fields
* add v2 test
* add response builder
* add test for update from server
* add test for update containing only changed resources
* clang-format
* fix build
* add test for resource not existing upon subscription
* add test for stream closed by server
* add test for multiple watchers for the same resource
* add test for connection failure
* clang-format
* add test for resources wrapped in Resource wrapper message
* add test for resource validation failure
* add test for multiple invalid resources, and fix a case in XdsClient
* add test for validation failure for already-cached resource
* add test for server not resending resources after stream disconnect
* clang-format
* fix XdsClient to report channel errors to newly started watchers
* fix XdsClient to send cached errors/does-not-exists to newly started watchers
* fix watcher to ensure events arrive in the expected order
* fix tests
* clang-format
* add test for multiple resource types
* fix xds_cluster_e2e_test
* Automated change: Fix sanity tests
* cleanup
* add federation tests
* clang-format
* remove now-unnecessary XdsCertificateProviderPluginMapInterface
* code review comments
* simplify XdsResourceType::Decode() API
* XdsClient: add unit tests for XdsClusterResourceType
* add XdsClient with gRPC bootstrap config
* add LB policy tests
* started adding CertificateProvider tests
* update for recent API changes
* fix merge bugs
* xDS resource validation: identify extensions by type_url instead of name
* fix build
* migrate to ValidationErrors
* add xds_common_types_test
* finish TLS tests and add LRS tests
* move ScopedExperimentalEnvVar to its own library and remove redundant e2e tests
* add circuit breaking and outlier detection tests
* add validation to outlier detection LB policy parsing
* clang-format
* Automated change: Fix sanity tests
* fix signedness
* fix sanity
* xDS: implement xds_wrr_locality LB policy and return xDS LB config from XdsClient
* fix unused parameter
* fix sanity
* fix test
* Automated change: Fix sanity tests
* fix aggregate cluster bug
* Automated change: Fix sanity tests
* absl::make_unique -> std::make_unique
* fix sanity
* fix sanity
* iwyu
* iwyu
* update code for XdsResourceTypeImpl changes
Co-authored-by: markdroth <markdroth@users.noreply.github.com>
* begin c++
* Automated change: Fix sanity tests
* progress
* progress
* missing-files
* Automated change: Fix sanity tests
* moved-from-stats
* remove old benchmark cruft, get tests compiling
* iwyu
* Automated change: Fix sanity tests
* fix
* fix
* fixes
* fixes
* add needed constructor
* Automated change: Fix sanity tests
* iwyu
* fix
* fix?
* fix
* fix
* Remove ResetDefaultEventEngine
Now that it is a weak_ptr, there's no need to explicitly reset it. When
the tracked shared_ptr is deleted, the weak_ptr will fail to lock, and a
new default EventEngine will be created.
* forget existing engine with FactoryReset
* add visibility
* fix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Co-authored-by: AJ Heller <hork@google.com>
* Re-add compiler protos to grpcio-tools
* Attempt to break
* Actually fail if protobufs don't compile. Who wrote this thing?
* Add an actual test
* yapf
* Whoops. Thought I'd reverted you
* Please the formatter gods
* Support Python 3.11
* Update build images for 3.11
* Whoopsie
* The architecture of this thing is garbage
* Silence ownership warning
* Account for change in git behavior
* Fix directory
* I am in great pain
* Update Windows and arm linux
* Agh
* Clean up
* [cleanup] Remove profiling timers
- nobody has used this system in years
- if we needed it, we'd probably rewrite it at this point to be something more modern
- let's remove it until that need arises
* fix
* fixes
A (currently) pthread_atfork-based fork support mechanism, allowing EventEngines - or any other object that wants to implement the Forkable interface - respond to forks.
* Rename the default EventEngine headers
Small cleanup. This code hasn't been related to factories for a month or
two.
* ensure only one target contains default_event_engine.h
* src + hdr in same target
* include guards
* Update third_party/protobuf
* run tools/distrib/python/make_grpcio_tools.py
* regenerate protos for ruby, php
* update build_handwritten.yaml
* regenerate projects
* Build - Use :well_known_type_protos instead of :well_known_protos
* Fix target
* Update upb
* Update Python for Protobuf 4.21 (#140)
* Update protobuf dependency on grpcio-tools
* Off by one
* Drop python 3.6 support
* Try upgrading pip
* And in the other script
* Try to figure out if we're compatible with abi3
* See what we've already got installed
* Update the requirements.txt file I didn't know existed
* And here too
* See what's installed
* Let's try that again
* Remove
* Try to confirm version
* Let me see the generated code
* Fix non-Bazel test runner
* Work for all test directories
* Regenerate example protos
* Clean up
* Generate .pyi files
* Fix type checking and linting
* Exclude pyi files from isort
* Upgrade to 3.21.4
* Update iwyu to get around messy protobuf IWYU rules
Co-authored-by: Richard Belleville <gnossen@gmail.com>
This is a partial fork of the windows iomgr code - specifically the IOCP and Socket pieces - with some improved architecture and encapsulation. And the start of a WindowsEventEngine.
Once this code is used in a gRPC TCP context, I imagine a few issues will shake out. Also, getting sanitizers set up with MSVC will take a bit of work (see a commit referencing abseil and MSVC bugs to hack around).
I forked the IomgrEventEngine's posix poller interfaces in the hope of negotiating compatibility between the platforms, but the interfaces diverged a fair bit, and I'm doubtful we'll be able to use these "pollers" generically in the same TCP code. Reunification might not happen, and that's probably fine, we'll see how similar the TCP code looks once it's fleshed out.
I also extracted the IomgrEventEngine's timer piece into a separate component, usable by both engines.
* xDS: upgrade the commit of the xDS protos that we're using
* Automated change: Fix sanity tests
Co-authored-by: markdroth <markdroth@users.noreply.github.com>
This change includes:
* adding a cert file path for MacOS
* updating related test to run on MacOS too
* s/linux/supported/ since this now includes more platforms
* regenerating files affected by the name change
* [Aio] Ensure Core channel closes when deallocated
* Keep channel object alive
* Let Multicallable objects hold the reference to Channel
* Make YAPF happy
This code is not compiled by default and has fallen out of sync with the
rest of the codebase. There's a good chance it won't be used, given our
current work to use an iomgr-drive EventEngine instead.
This code will continue to live in git history, should we need to bring
pieces of it back.
* Initial skeleton for outlier detection
* fixing code review comments (modifying child policy)
* Skeleton and all tests passing except for 1
* small code review comments fix
* Adding the parsing of the policy in cds and put it in discovery
mechansim json format
* Parsing outlier detection json policy from parent
* Adding parsing of the updates
* Added Subchannel wrapper and watcher wrapper: and all states pass
through and all tests still pass
* added framework to do eject and uneject
* fixing code review comments
* restore a test
* fixing code review comments
* taking care of code review comments
* removing debug code and rebuild build files
* fixing according to code review comments
* fixing code review comments
* Adding address to subchannel map
* addressing code review comments
* adding call counter
* Refcount SubchannelState (in the map) and store them in Subcahnnel Wrapper
* fixing counterss
* Call counter and tracker skleton added
* Call counter
* addressing code review comments
* addressing code review comments
* Added CallCounter and timer
* fixing sanity; but more importantly: taking out timer temporarly as it
was causing test failures.
* sanity
* fixing according to code review comments
* addressing code review comments
* all algorithms implemented
* addressing code review comment about starting the timer
* protect private vars
* small fix
* Added one more corner case
* fixing EjectionTimer
* Fixing according to code review suggestions.
* fixing according to code reveiw comments
* taking care of code review comments
* fixing sanity issues
* Adding proto to tests
* First test
* Fixing according to code review comments
* Tests all working now
* fixing a crash
* fixing build files
* fixing sanity
* sanity
* Simplifying tests
* merge and update
* format
* sanity and format
* Fixing asan error
* fixing parsing logic and error handling
* 6 more tests done
* Added verifying unejection to tests
* Added all the tests
* fixing according to code review comments
* fixing asan and ubsan
* Fixing tests according to code review comments
* Added both algorithm tests
* added percentage enforcement tests
* fixing tsan error
* keeping debug, but fix warning
* remove debugs
* fixing IWYU and build errors after
* test comments change only but very important
* fixing code review comments
* one more refactorying of util function
* Removed debugs and added one more helper method
* one more logic fix
* Fixing last bit of code review comments and added disable tests
* fixing code review comments
* fixing IWYU
* sanity format
* protecting the feature with environment var:
registering policy and generating policy
* added a todo according to code review comments
* fixing a clang finding at import time
* build fix after synching to latest
Rollforward with TCP connect handshaker again(#29111) after fixing broken internal targets.
The changes needed were just visibility changes to the handshaker and the http_connect_handshaker libraries as they are used internally.
* Move unnecessary absl includes from metadata_batch.h
metadata_batch.h is a widely used header, and so any includes it takes
that are for its internal use can negatively impact our build times.
* missing file
* build
* review feedback
Currently the tcp connect is performed in chttp2_connector before the handshaking is triggered. For
use cases where the application wants to perform business logic before
the tcp connection, this is problematic. By moving the TCP connect into
its own handshaker and registering it by default at the beginning, this
allows applications to add a new handshaker at the beginning allowing
handshaker logic before a TCP connect.
This approach has the advantage of slightly simplifying the logic in
tcp_connect_handshaker and httpcli as tcp_connect/callback can be
removed.
As the TCP connect needs parameters like resolved_addr,
interested_parties, a new struct called connection args is created as a
member for Handshaker Args.
For server handshakers most of the arguments here are not directly
useful, other than the deadline.
* WIP: add OOB backend metric API for LB policies
* fix some includes
* minor fixes
* picking this up again...
* more WIP
* health checking: cancel stream if response message fails to parse
* basic structure in place, but still have synchronization issues to address
* ORCA: implement ORCA RPC service for OOB backend metric reporting
* fix unused parameter error
* gen_upb_api
* add missing build deps
* increase test timing fudge factor
* add missing copyright header
* fix build and locking problems
* clang-format
* document API
* buildifier
* add test, but doesn't build yet
* new test working, but broke existing test, and need to fix server API
* don't register as a generic service
* update test for new orca service registration API
* fix build
* sanitize
* report interval defaults to min interval
* add channel trace event on UNIMPLEMENTED
* don't regenerate the response proto unless something changed
* add missing build dep
* fix comment
* Initial unit testbench
* Tentative fix
* Reduce scope of fix. Clean up test
* Formatting
* Protect writing to RPC status store
* Don't start client until server is running
* Typo
* Remove redundant log
* More formatting
* Review comments
* [PSM interp] Set hostname in initial metdata for Python servers
* DO-NOT-MERGE: hijact psm-security-python for one test case
* Revert "DO-NOT-MERGE: hijact psm-security-python for one test case"
This reverts commit 17b8af9ab9.
* Metadata is a tuple of tuple not a tuple
* Initial structure for RLS
* Adding and building the proto to parse the Any proto for the plugins
* re-org
* Parsing the plugin
* Parsing more into json
* Parsed proto to json
* small cleanup
* Adding prefix
* Added new rls_experimental policy
* build files
* Fixing according to code review comments
* code review comments
* Adding sym changes
* adding action name check
* fixing code review comments.
* fixing unused var error
* clean up
* fixing code review comments
* fixing code review comments
* fixing according to code review comments.
* Remove unnecessary include
* small fix
* generate more, hard-code less
* Moving to using absl::variant
* absl::string_view and absl::variant of vector of std::string are not
playing nice together.
* fixed variant
* Using absl::variant now
* Checkint used plugins
* Refactor Parsing code and separating out Parsing of the plugin
* Fixing code review comments
* code review comments
* fixing code review comments.
* Addressing code review comments
* First end-to-end test
* generated build files
* commit generated files via tools/codegen/core/gen_upb_api.sh
* Fixing rls policy parsing tests
* Restore checks for the test server
* Refactor rls_server
* added keys to rls request
* fixing small logic error
* Complete the test using all the keys
* Separating out RLS test and rls_server thread
* sanity errors
* generated build files
* Complete the rest of the tests and sanity cleanup
* fixing code review comments: using upb_JsonEncode now!
* fixing code review comments
* fixing code review comments
* Fixing code review comments
* misisng fix
* simplifying tests
* simplify tests 2
* Linking in the correct proto for rls_config
* restore metadata check
* Add disable test
* Fixing RLS test and removing environment var that is no longer necessary
* Fixing "Wrong type" type of tests after json parsing change to accept
STRING for number
* adding json_encode.h/c to src/upb/gen_build_yaml.py and generate
necessary files.
* Fixing un-used var error
* fixing sanity errors
* Fixing the upb encoding buffer
* Fixing code review comments.
* Adding nack test for unkonwn plugin proto
* Last bit of code review comments
* fixing unused variable
* WIP
* Add gevent test suite run under Bazel.
* Fix things up
* Yapf
* Fix up Bazel files
* Make py_grpc_test fancier
* Attempt to fix Windows RBE
* Attempt to kick GitHub
* Fix Python 2 runs
* Yet more fixes
* And the patch file too
* I am an idiot
* Mark gevent tests flaky
* Try to make rules_python more tolerant
* Typo
* Exclude reconnect test from gevent
* Remove unnecessary parts of patch
* Buildifier
* You saw nothing
* isort
* Move py_grpc_test to an internal-only file
* Review comments
* More reviewer comments
* Review
* Add initial changes for gevent
* WIP. Run completion_queue_next in a threadpool
* WIP.
* WIP
* Re-remove skip annotation
* Finally working
* Reactivate tests
* Clean up
* Move C++ threading utilities to grpc.pxi
* Unbreak sync stack
* Refix test flake
* WIP. Trying to get things working properly
* Move test stuff to test runner
* Clean up
* Can't handle exceptions if you don't compile with exceptions
* Remove debug stuff unintentionally left in
* Add greenlet switch loggging and fix threading issue
* Only run a greenlet scheduling greenlet when there are open channels
* Format
* Add threadpool modifications to old runner
* And actually import gevent
* Move XdsChannelCreds to CoreConfiguration
* move xDS channel creds files to src/core/lib/security/credentials/xds
* Change back to returning a RefCountedPtr.
* make remove "xds_" from xds_channel_* files.
* Renamed to address comments.
* clang fix
* Fix another clang error
Partially collapse `event_engine_factory.cc` into `event_engine.cc`. Add a
new function `DefaultEventEngineFactory` which is used to set a default
event engine factory at link time, separate from the factory that can be
set at run time. Implemenet this function in
`default_event_engine_factory.cc`.
This allows alternative default event engine factories to be implemented
without requiring the duplication of the implementations of
`SetDefaultEventEngineFactory`, `CreateEventEngine`, and
`GetDefaultEventEngine`.
* Adding TLS Key export logic to core and c++ wrappers
* Adding and end2end cpp tls key export test and updating broken test due to interface changes
* regenerate projects
* updating tls key export core logic with addition of APIs to grpc_security.h
* undoing changes to tls_security_connector_test
* regenerate projects
* changing the logging format enum name as per GRFC comments
* regenerate projects
* removing some commented code
* updating changes as per review comments
* adding GRPCAPI annotations to functions defined in grpc_security.h
* regenerate projects
* fixed some code styling issues
* removing grpc_security.h include from tls_credentials_options.h
* updating files as per review comments
* minor fixes
* moving some code around
* removing key log format from tls session key log config and converting it to a simple string
* regenerate projects
* fixing mistakes in recent merge with master
* regenerate projects
* regenerate projects
* fixing some distrib and snity errors
* fixing formatting errors
* fixing more sanity checks and raising supported openssl versions to 1.1.1
* updating min supported openssl version to 1.1.1
* updating min supported openssl version in tls_key_export_test
* updating test to fix incorrect vector initialization
* updating as per latest comments
* fixing sanity checks
* addressing review comments
* fixing sanity checks
* fixed c++ comment style
* Automated change: Fix sanity tests
* fixing review comments
Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
Eliminate slice interning, and structures in slices to support it.
Reduces grpc_slice_refcount from 40 bytes (+ a required 8 bytes elsewhere) to 16 bytes.
Removes a pointer dereference for every slice ref/unref.
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
This patch introduces a factory to allow supporting custom xDS channel
creds. Three types currently supported (fake, insecure, google_default)
are registered by default for backward-compatibility.
* Fix the distutil install issue
* Try to move setuptools install forward
* Try pin the distribution tools
* Enforce local distutil
* Fix script order of definition
* Add comment
* Try to fix the setuptool monkey patch
* Clean-up imports for commands.py
* Pin 59.6.0 instead of 59.7.0 (incomplete release)
* WIP
* introduce XdsResourceType API and change Listener parsing to use it
* converted RouteConfig parsing
* convert cluster and endpoint parsing
* cleanup
* clang-format
* attempt to work around compiler problems
* move XdsResourceType to its own file, and move endpoint code out of XdsApi
* move cluster parsing to its own file
* move route config parsing to its own file
* move listener parsing to its own file
* clang-format
* minor cleanup
* plumbed XdsResourceType throughout XdsClient
* a bit of cleanup
* more cleanup
* construct full resource names before calling XdsApi::CreateAdsRequest()
* remove some unneeded code
* clean up includes and have XdsResourceType initialize the upb symtab
* more cleanup of unnecessary code
* more cleanup
* update comment
* clang-format
* add missing virtual dtor
* fix build
* remove comment
* add missing virtual dtor
* introduce XdsResourceType API and change Listener parsing to use it
* converted RouteConfig parsing
* convert cluster and endpoint parsing
* cleanup
* clang-format
* attempt to work around compiler problems
* move XdsResourceType to its own file, and move endpoint code out of XdsApi
* move cluster parsing to its own file
* move route config parsing to its own file
* move listener parsing to its own file
* clang-format
* minor cleanup
* remove comment
* add missing virtual dtor
* Fix status code for resource exhaustion
* Revert "Revert "Move a bunch of slice typed metadata to new system (#28107)" (#28208)"
This reverts commit 7717587063.
* fix test
* Check if memory owner available prior to polling it
The transport may drop the memory owner during its destruction sequence
* tcp_fix
* Revert "Revert "New resource quota integration (#27643)" (#28014)"
This reverts commit 0ea2c37263.
* clang-format
* fix-path
* fix
* Rename the source files for ChannelArgsEndpointConfig
Previous naming was non-descript
* generate_projects
* add missing file to BUILD, and generate_projects.sh
* correct include guards
* Reintroduce the EventEngine default factory
An application can provide an EventEngine factory function that allows
gRPC internals to create EventEngines as needed. This factory would be
used when no EventEngine is provided for some given channel or server,
and where an EventEngine otherwise could not be provided by the
application. Note that there currently is no API to provide an
EventEngine per channel or per server.
I've also deleted some previous iterations on global EventEngine and
EventEngine factory ideas. This new code lives in a public API, and
coexists with iomgr instead of being isolated to an EventEngine-specific
iomgr implementation.
* add proper namespaces, and fix description
* put factory functions in their own file (for replaceability)
* add synchronization
* generate_projects.sh
* extract event_engine_base and event_engine_factory targets
Also separate iomgr/event_engine files in the BUILD, with comments
* gpr_platform
* move all EE factory declarations to event_engine_base
Makes internal hackery easier.
* add missing deps
* reorder dep alphabetically
* comment style change
* Fix all lint errors in repo.
* Use strict buildifier by default
* Whoops. That file does not exist
* Attempt fix to buildifier invocation
* Add missing copyright
* Support generic_xds_config in GCE/GKE frameworks and Python tests
* Correct the field name
* Address comments
* Add new CSDS support to the regular K8s testcases
* Support incomplete CSDS config dump
* ChannelStackModifier class
* Regenerate projects
* clang-tidy
* Use CoreConfiguration for inserting xDS HTTP filters
* New lines
* Move test to test/core/xds
* Add comment for placement of filter stack
* Fix sanity
* Fix memory leak - destroy builder
* Don't build xds_channel_stack_modifier_test on non-bazel build systems due to census dependency
* Move most py_proto_library macro contents into rule.
The _generate_pb2_src rule is modified to produce a valid PyInfo
provider that can be depended on directly by py_.* rules.
Keeping an intermediate macro in place increases complexity without
adding any real value.
* Move most py_grpc_library macro contents into rule.
The _generate_pb2_grpc_src rule is modified to produce a valid PyInfo
provider that can be depended on directly by py_.* rules.
Keeping an intermediate macro in place increase complexity without
adding any real value.
* Remove support for additional plugins.
It's easy enough for a plugin author to build their own rule-set, and it
adds extra unused (and untested) complexity to gRPC's rules.
* Rewrite py_proto_library to track dependencies correctly.
* Propagate Python dependencies of py_grpc_library rules to consumers.
* Add test coverage for aspect-based py_proto_library.
This commit is part of the effort to create binder channel with
GRPC_CLIENT_CHANNEL type.
The resolver will be used during name resolution, and the result will
later be used to identify the corresponding endpoint binder in
SubchannelConnector.
Besides the unit test, this change is tested with other changes locally
end to end on real device.
* Don't leave pending tasks on the asyncio queue
The results of these pending tasks are not needed, leaving them
on the queue grows the size of the queue until the call completes.
This fix slows the growth of the memory in the test example.
* Address 'leaking' Futures from cygrpc
Cancelling unneeded Tasks is not sufficient as this leaves behind
cancelled Futures in the cygrpc layer, which still occupy memory.
Instead, avoid creating unneeded tasks in the first place.
* Address review comments
1. Ignore unused return values
2. Fix formatting
* RLS LB policy plugin implementation
* Add RLS tests
* rls proto
* generate projects
* Fix adding child policy to child policy map
* Fix issues
* Fix rotten bits
* generate projects
* revert some strange changes
* First batch of fixes
* second batch of addressing comments
* generate_projects
* 3rd batch of fixes
* generate_projects
* 4th batch of fix
* 5th batch of fix
* Empty commit to trigger github PR refresh
* Add insecure build
* generate project
* update picker in a closure
* mutex guard audit
* clang-format
* Aggregate child policy states
* Use OrphanablePtr on ChildPolicyOwner()
* More fixes on comments
* Remove include grpc_security from rls.cc
* Fix key builder test as we removed RlsFindPathFromMetadata
* Update rls proto and add multi-target capability
* code changes to fix build after merging master
* remove support for insecure builds
* WIP
* add LB policy metadata lookup API
* add API for creating errors from C++ strings
* add missing include
* finished updating JSON parsing
* use DualRefCounted<> for ChildPolicyWrapper
* use grpc_core::Mutex instead of std::recursive_mutex
* add lock annotations and fix some lifetime issues
* misc fixes and cleanups
* simplify child policy connectivity state machine
* add comment about hopping into ExecCtx
* don't use wait_for_ready for RLS call
* extraKeys and constantKeys support
* add lock annotations for child policy wrapper connectivity state
* improve logging
* simplify child policy state handling in picker
* use C++ style comments
* remove rls_config.proto, since it's not needed
* rewrote tests
* fix build
* fix copyright headers
* clang-format
* appease clang-tidy
* remove unnecessary dependendency on grpclb
* buildifier
* remove illegal term
* remove unneeded includes
* clang-format
* fix clang-tidy
* fix build
* move class declaration into .cc file
* clang-tidy again
* fix build
* reorganize code and misc cleanups
* clang-format
* fixed a bunch of asan bugs; still some left to debug
* fix asan problem
* make test work on IPv6-only machines
* move LRU list handling into Cache::Entry and add lock annotations
* use preincrement instead of postincrement
* fix deadlock
* add more FIXMEs
* clean up backoff timer logic
* fix build from merge
* clang-format
* fix include path to work on podspec builds
* clean up picker logic
* clang-format
* update rls.proto
* populate reason and stale_header_data in RLS request
* improve logging and fix some cache size type issues
* clang-format
* add some TODOs about a better way to expose channel creds to LB policies
* centralize handling of channel's default authority
* clang-format
* fix backup poller bug
* fix handling of call creds and authority on RLS channel
* remove unused params
* clang-format
* use two-phase update for child policies to avoid deadlocks
* fix portability issue
* minor cleanups
* update for change in grpc_error_get_status() API
* change test to store RLS requests and responses in proto form
* clang-format
* account for test slowdown factor in timeouts
* fix tsan failure and channelz linkage
* fix RLS authority death test
* fix test to not try to connect to target that should not work
* remove illegal term
* cacheSizeBytes is a required field
* add missing BUILD deps from merge
* add tests for connectivity state reporting and fix bugs found
* fix BUILD sanity
* buildifier
* fix BUILD package path
* fix bugs related to child policy lifetime and updates
* remove unnecessary srand() call from grpc_init()
* add test for two cache entries with the same target
* update rls.proto
* change RLS config parser test to support GRPC_ERROR_IS_ABSEIL_STATUS
* update upb codegen
* fix include path
* add env var guard
* avoid duplication of CountedService
* generate_projects
* add upper limit for cacheSizeBytes
* fix build
Co-authored-by: Muxi Yan <mxyan@google.com>
* WIP
* Add gevent test suite run under Bazel.
* Fix things up
* Yapf
* Fix up Bazel files
* Make py_grpc_test fancier
* Attempt to fix Windows RBE
* Attempt to kick GitHub
* Fix Python 2 runs
* Yet more fixes
* And the patch file too
* I am an idiot
* Mark gevent tests flaky
* Try to make rules_python more tolerant
* Typo
* Exclude reconnect test from gevent
* Remove unnecessary parts of patch
* Buildifier
* You saw nothing
* isort
* Move py_grpc_test to an internal-only file
* Review comments
* More reviewer comments
* Review
* make error independent
* Separate grpc_error into its own library
This is forward work to move Closure, ExecCtx into their own libraries
in order to make use of them in the activity code for resource quota
wakeups.
* Automated change: Fix sanity tests
* fixes
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* separate slice into internal and refcount, with refcount being the minimal base
* python3-ize
* Separate slices from mdelems for static data
Allows us to separate a small build target with just some slice basics
from the rest, which will help simplify the build system longer term.
Additionally exposed an opportunity to eliminate an init function.
Should also help in a month or so when it's time to jettison interned
metadata and slices entirely.
* fix compilation
* Automated change: Fix sanity tests
* fixes
* fixfixfix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Disable grpc_status_test for sync stack in Python2
* Correct the if condition
* Also disable this test in Bazel
* Clean the unnecessary import in BUILD
* Add Python 3.10.0rc1 binary wheels
* Drop Python 3.5 artifacts
* Document the drop of 3.5
* Fix the wrong pip pointer
* Update manylinux2014 to a newer version, remove 3.5 distribtest
* Update manylinux aarch64 to see if the absl error go away
* Use the preferred alias
* Allow different wheel library to produce different tag order
* Remove unused shell var and log produced wheels
* Use copy instead of move
* Make bash happy about the wildcard
* Upgrade the debian image to use 3.5+ Python
* Polish the comments for the Dockerfiles
* Implement type safety for TLS
This is mostly free when compiler support is available, but requires
careful templating when implemented using pthread.
Significantly slimmed the tls.h interface; it now only defines the "TLS
keyword" for each supported compiler, delegating enforcement of correct
usage (i.e. must be static) to the compiler itself.
Implemented implicit conversion for the pthread wrapper so it can be
used (mostly) the same as native support. Notable exception to this is
that static_cast<void*> is needed when printing a pointer stored in TLS
as %p.
* Use GPR_THREAD_LOCAL macros consistently
* Bump Envoy and related submodules to latest
* Update googleapis
* Update scripts for newer submodules
* Update udpa
* Add opentelemetry
* Python changes for opentelemetry
* Update udpa in check_submodules
* Add opentelemetry to check submodules
* Regenerate upb files
* Add new proto dependency to upb-gen and Bazel
* Regenerate project
* Add isort_code.sh to sanity tests
* Run tools/distrib/isort_code.sh
* Fine tune the import order for relative imports
* Make pylint and project generation happy
* Fix a few corner cases
* Use --check instead of --diff
* The import order impacts test result somehow
* Make isort print diff and check output at the same time
* Let tools/run_tests/python_utils be firstparty library
* Run isort against latest HEAD
* Remove Python 2.7 binary wheels
* Make sure pip won't pick-up newer versions
* Attempt to fix the distribtest
* Fallback to virtualenv==16.7.9 for 34 and 35
* Remove 34 from binary wheel distribtests
* fix: If client request iterators raise exceptions, _consume_request_iterator should not dump its stack to stderr
fixes: #25339
* fix: use format_exc
* fix: linter
* fix: lint round 2
This code adds an iomgr implementation that's backed by an EventEngine. This uses the EventEngine API alone, and separate work will introduce an EventEngine prototype to plug into it.
See also drfloob#1: @nicolasnoble has a pull request against this branch, implementing the libuv-based EventEngine. One goal here is to implement the iomgr code such that it can be merged independently without affecting normal builds.
This implementation can be built using bazel build --cxxopt='-DGRPC_USE_EVENT_ENGINE' :all
Some shortcuts are being taken to get a working, testable version of the engine. EventEngines are not pluggable, for example.
* support user provided audience in gdc and jwt
* fix 1st round of comments
* fix ruby and php to use new GDC API
* fix python clang issue
* address 2nd round of comments
* fix string_view issue
* remove length param in string_view constructor
* Fix metadata plugin for concurrent invocations
* Disable broad-except lint error
* Use WaitGroup instead of threading.Event
* Fix typo
* Use only a single channel
* Yapf
* Handle gevent exception in gevent poller
Currently the gevent poller ignores exceptions raised by
`gevent.wait()`, which causes greenlets to be unkilable while waiting.
This change handles exceptions raised while waiting in the gevent
poller, cancels the gRPC call and propagates the error back to the
application.
Co-authored-by: Kostis Lolos <klolos@arrikto.com>
* Fix imports in header files
* Lint gevent tests
* Set grpc event type to GRPC_QUEUE_SHUTDOWN upon cancel error
To prevent `grpc_completion_queue_next()` to be called indefinitely when
the queue is shut down.
* Remove unnecessary `except *`
* Improve gevent tests
* Format code
* Remove unnecessary import
Co-authored-by: Kostis Lolos <klolos@arrikto.com>
* Increase core logging verbosity
* Make repo URL injectable
* Apparently that's not allowed
* Use ipv4 in interop server
* Add CSDS to Python interop client
* Revert CI script change
* Scale back logging
* Move env variable to dockerfile
* And Python too
* Add grpcio-csds pacakge
* Remove unused file
* Fix the proto import path issue
* Update the CSDS package and xds-protos for PY2
* Make tests happy
* Fix Bazel proto dependency
* Add Python2 tests for CSDS
* Expose code and details from context on the server side
This makes them accessible to server-side interceptors.
* Complete synchronous implementation
* Add experimental API notes
* Complete async implementation
* Revert bogus copyright change
* Fix metadata tests for py2.7
* Add new test to tests.json
* Check error codes in tests
* Bump autogenerated Python version
* Add channelz to security client
* Make client changes. Some config needs to be reverted
* Add missing security field to channelz Socket
* Add test
* Fix test
* Remove whitespaces for windows
* Remove local_certificate check from PSM security tests
* Byte pack Python channelz IPs
* Move address packing to Core
* WIP
* Unbork security tests
* Python interop server
* Turn up server logging
* Clean up PR
* Clean up some more
* Yapf
* Fix up docker images
* Swap fillllllles!
* Add more robust boolean arg parsing
* Use bool_arg parsing in server as well
* Add copyright
Co-authored-by: Yash Tibrewal <yashkt@google.com>
* pass XdsClient to LB policies via channel args
* add channel arg for overriding bootstrap config on a per-channel basis
* change tests to use new channel arg approach -- currently failing for server-side tests
* use the same channel args approach on the server side
* clang-format
* fix CircuitBreakingMultipleChannelsShareCallCounter test
* fix XdsEnabledServerTest test
* add TODO
* clang-format
* generate_projects
* fix clang-tidy
* fix build
* attempt to fix python
* rename ChannelData to ClientChannel
* make ClientChannel class definition public
* move retry code to its own filter
* move LB call factory method to ClientChannel class
* move dynamic termination filter out of ClientChannel class
* update comments
* remove retry parsing from client channel service config parser
* fix clang-tidy
* fix service_config_test
* clang-format
AJ Heller