This change includes:
* adding a cert file path for MacOS
* updating related test to run on MacOS too
* s/linux/supported/ since this now includes more platforms
* regenerating files affected by the name change
* [resource_quota] Periodic update tracker
Periodic update type that tries to do updates at some measurable
timescale without polling the current time every tick, instead trying to
approximate how many ticks will be required to fill the desired
duration.
The intent here is that we use this in RQ to donate memory back
periodically whilst keeping timer checks off the fast path.
* fix
* Automated change: Fix sanity tests
* review feedback
* fix
* fix
* review feedback
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* implement UniqueTypeName API
* convert security code to use UniqueTypeName
* change subchannel data producer API to use UniqueTypeName
* sanitize
* add missing build dep
* fix credentials_test
* fix certificate_provider_store_test
* fix tls_security_connector_test
* attempt to fix windows build
* avoid unnecessary allocation
* work around MSVC 2017 bug
* sanity
* change factory to not be templated
* fix sanity
* fix bug in chttp2 connector that used server creds instead of channel creds
* add missing build dep
* simplify API
* Initial skeleton for outlier detection
* fixing code review comments (modifying child policy)
* Skeleton and all tests passing except for 1
* small code review comments fix
* Adding the parsing of the policy in cds and put it in discovery
mechansim json format
* Parsing outlier detection json policy from parent
* Adding parsing of the updates
* Added Subchannel wrapper and watcher wrapper: and all states pass
through and all tests still pass
* added framework to do eject and uneject
* fixing code review comments
* restore a test
* fixing code review comments
* taking care of code review comments
* removing debug code and rebuild build files
* fixing according to code review comments
* fixing code review comments
* Adding address to subchannel map
* addressing code review comments
* adding call counter
* Refcount SubchannelState (in the map) and store them in Subcahnnel Wrapper
* fixing counterss
* Call counter and tracker skleton added
* Call counter
* addressing code review comments
* addressing code review comments
* Added CallCounter and timer
* fixing sanity; but more importantly: taking out timer temporarly as it
was causing test failures.
* sanity
* fixing according to code review comments
* addressing code review comments
* all algorithms implemented
* addressing code review comment about starting the timer
* protect private vars
* small fix
* Added one more corner case
* fixing EjectionTimer
* Fixing according to code review suggestions.
* fixing according to code reveiw comments
* taking care of code review comments
* fixing sanity issues
* Adding proto to tests
* First test
* Fixing according to code review comments
* Tests all working now
* fixing a crash
* fixing build files
* fixing sanity
* sanity
* Simplifying tests
* merge and update
* format
* sanity and format
* Fixing asan error
* fixing parsing logic and error handling
* 6 more tests done
* Added verifying unejection to tests
* Added all the tests
* fixing according to code review comments
* fixing asan and ubsan
* Fixing tests according to code review comments
* Added both algorithm tests
* added percentage enforcement tests
* fixing tsan error
* keeping debug, but fix warning
* remove debugs
* fixing IWYU and build errors after
* test comments change only but very important
* fixing code review comments
* one more refactorying of util function
* Removed debugs and added one more helper method
* one more logic fix
* Fixing last bit of code review comments and added disable tests
* fixing code review comments
* fixing IWYU
* sanity format
* protecting the feature with environment var:
registering policy and generating policy
* added a todo according to code review comments
* fixing a clang finding at import time
* build fix after synching to latest
* move some code around
* remove num_backends parameter from XdsEnd2endTest
* remove use_xds_enabled_server param from XdsEnd2endTest
* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest
* remove client_load_reporting_interval_seconds param from XdsEnd2endTest
* start moving CreateAndStartBackends() into individual tests
* finish moving CreateAndStartBackends() into individual tests
* remove unused variable
* remove SetEdsResourceWithDelay
* fix test flake
* clang-tidy
* clang-format
* move test framework to its own library
* fix build
* clang-format
* fix windows build
* rename TestType to XdsTestType
* move BackendServiceImpl inside of BackendServerThread
* clang-format
* move AdminServerThread to CSDS test suite
* move RLS tests to their own file
* remove unnecessary deps
* generate_projects
* move some code around
* remove num_backends parameter from XdsEnd2endTest
* remove use_xds_enabled_server param from XdsEnd2endTest
* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest
* remove client_load_reporting_interval_seconds param from XdsEnd2endTest
* start moving CreateAndStartBackends() into individual tests
* finish moving CreateAndStartBackends() into individual tests
* remove unused variable
* remove SetEdsResourceWithDelay
* fix test flake
* clang-tidy
* clang-format
* move test framework to its own library
* fix build
* clang-format
* fix windows build
* move fault injection tests to their own file
* rename TestType to XdsTestType
* move BackendServiceImpl inside of BackendServerThread
* clang-format
* generate_projects
* appease clang-tidy
* move AdminServerThread to CSDS test suite
* remove unnecessary deps
* generate_projects
* don't mark test as flaky
* Revert "Revert "ORCA: implement ORCA RPC service for OOB backend metric reporting (#29215)" (#29351)"
This reverts commit 71b355624f.
* move ORCA service to its own BUILD rule
* ORCA: implement ORCA RPC service for OOB backend metric reporting
* fix unused parameter error
* gen_upb_api
* add missing build deps
* increase test timing fudge factor
* add missing copyright header
* buildifier
* don't register as a generic service
* report interval defaults to min interval
* don't regenerate the response proto unless something changed
* use INTERNAL for proto parsing failure
* use absl::Duration in public API
* Refactor end2end tests to exercise each EventEngine
* fix incorrect bazel_only exclusions
* Automated change: Fix sanity tests
* microbenchmark fix
* sanitize, fix iOS flub
* Automated change: Fix sanity tests
* iOS fix
* reviewer feedback
* first pass at excluding EventEngine test expansion
Also caught a few cases where we should not test pollers, but should
test all engines. And two cases where we likely shouldn't be testing
either product.
* end2end fuzzers to be fuzzed differently via EventEngine.
* sanitize
* reviewer feedback
* remove misleading comment
* reviewer feedback: comments
* EE test_init needs to play with our build system
* fix golden file test runner
Co-authored-by: drfloob <drfloob@users.noreply.github.com>
* Revert "Revert "HTTP2: Add graceful goaway (#29050)" (#29187)"
This reverts commit 882f64e376.
* Make sure fds are added to pollsets before the main test contents
* Call finalization for promises
* comment
* split out and test
* dont use promise_detail:: directly
* fix
* Automated change: Fix sanity tests
* fix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Move XdsChannelCreds to CoreConfiguration
* move xDS channel creds files to src/core/lib/security/credentials/xds
* Change back to returning a RefCountedPtr.
* make remove "xds_" from xds_channel_* files.
* Renamed to address comments.
* clang fix
* Fix another clang error
* HTTP2: Add GrpcNetworkStreamState metadata for calls that are not sent on wire and for those that are not seen by server
* Generate projects
* clang-tidy
* Fix test
* clang-tidy
* Add a negative test
* Fix for windows
This avoids having to do a cherry-pick import, and is harmless since
there are no dependencies yet on the EventEngine. This test will be
re-enabled shortly after both the import and related changes are
finished.
* Adding TLS Key export logic to core and c++ wrappers
* Adding and end2end cpp tls key export test and updating broken test due to interface changes
* regenerate projects
* updating tls key export core logic with addition of APIs to grpc_security.h
* undoing changes to tls_security_connector_test
* regenerate projects
* changing the logging format enum name as per GRFC comments
* regenerate projects
* removing some commented code
* updating changes as per review comments
* adding GRPCAPI annotations to functions defined in grpc_security.h
* regenerate projects
* fixed some code styling issues
* removing grpc_security.h include from tls_credentials_options.h
* updating files as per review comments
* minor fixes
* moving some code around
* removing key log format from tls session key log config and converting it to a simple string
* regenerate projects
* fixing mistakes in recent merge with master
* regenerate projects
* regenerate projects
* fixing some distrib and snity errors
* fixing formatting errors
* fixing more sanity checks and raising supported openssl versions to 1.1.1
* updating min supported openssl version to 1.1.1
* updating min supported openssl version in tls_key_export_test
* updating test to fix incorrect vector initialization
* updating as per latest comments
* fixing sanity checks
* addressing review comments
* fixing sanity checks
* fixed c++ comment style
* Automated change: Fix sanity tests
* fixing review comments
Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
Eliminate slice interning, and structures in slices to support it.
Reduces grpc_slice_refcount from 40 bytes (+ a required 8 bytes elsewhere) to 16 bytes.
Removes a pointer dereference for every slice ref/unref.
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
This patch introduces a factory to allow supporting custom xDS channel
creds. Three types currently supported (fake, insecure, google_default)
are registered by default for backward-compatibility.
* Revert "Revert "user-agent metadata trait, also: grpc_core::Slice is born (#27770)" (#28108)"
This reverts commit 89d08dad9d.
* will it blend
* will it blend
* will it blend
* lcnagfmt
* sanity
* will it blend
* sleep @ end
* will it blend
* Revert "sleep @ end"
This reverts commit d6bca8ed3d.
* review feedback
* review feedback
* Check if memory owner available prior to polling it
The transport may drop the memory owner during its destruction sequence
* tcp_fix
* Revert "Revert "New resource quota integration (#27643)" (#28014)"
This reverts commit 0ea2c37263.
* clang-format
* fix-path
* fix
* Reintroduce the EventEngine default factory
An application can provide an EventEngine factory function that allows
gRPC internals to create EventEngines as needed. This factory would be
used when no EventEngine is provided for some given channel or server,
and where an EventEngine otherwise could not be provided by the
application. Note that there currently is no API to provide an
EventEngine per channel or per server.
I've also deleted some previous iterations on global EventEngine and
EventEngine factory ideas. This new code lives in a public API, and
coexists with iomgr instead of being isolated to an EventEngine-specific
iomgr implementation.
* add proper namespaces, and fix description
* put factory functions in their own file (for replaceability)
* add synchronization
* generate_projects.sh
* extract event_engine_base and event_engine_factory targets
Also separate iomgr/event_engine files in the BUILD, with comments
* gpr_platform
* move all EE factory declarations to event_engine_base
Makes internal hackery easier.
* add missing deps
* reorder dep alphabetically
* comment style change
This commit is part of the effort to create binder channel with
GRPC_CLIENT_CHANNEL type.
The resolver will be used during name resolution, and the result will
later be used to identify the corresponding endpoint binder in
SubchannelConnector.
Besides the unit test, this change is tested with other changes locally
end to end on real device.
The pool serves as a buffer for interaction between C++ and Java.
The buffer can let us avoid calling into Java code in channel connector
implementation (which has not been merged yet), simplifies interaction
between C++ and Java.
Temporary changes are made to channel_create.cc to keep the example apps
working but they will be rewrite after we start creating client channel
instead of direct channel.
* RLS LB policy plugin implementation
* Add RLS tests
* rls proto
* generate projects
* Fix adding child policy to child policy map
* Fix issues
* Fix rotten bits
* generate projects
* revert some strange changes
* First batch of fixes
* second batch of addressing comments
* generate_projects
* 3rd batch of fixes
* generate_projects
* 4th batch of fix
* 5th batch of fix
* Empty commit to trigger github PR refresh
* Add insecure build
* generate project
* update picker in a closure
* mutex guard audit
* clang-format
* Aggregate child policy states
* Use OrphanablePtr on ChildPolicyOwner()
* More fixes on comments
* Remove include grpc_security from rls.cc
* Fix key builder test as we removed RlsFindPathFromMetadata
* Update rls proto and add multi-target capability
* code changes to fix build after merging master
* remove support for insecure builds
* WIP
* add LB policy metadata lookup API
* add API for creating errors from C++ strings
* add missing include
* finished updating JSON parsing
* use DualRefCounted<> for ChildPolicyWrapper
* use grpc_core::Mutex instead of std::recursive_mutex
* add lock annotations and fix some lifetime issues
* misc fixes and cleanups
* simplify child policy connectivity state machine
* add comment about hopping into ExecCtx
* don't use wait_for_ready for RLS call
* extraKeys and constantKeys support
* add lock annotations for child policy wrapper connectivity state
* improve logging
* simplify child policy state handling in picker
* use C++ style comments
* remove rls_config.proto, since it's not needed
* rewrote tests
* fix build
* fix copyright headers
* clang-format
* appease clang-tidy
* remove unnecessary dependendency on grpclb
* buildifier
* remove illegal term
* remove unneeded includes
* clang-format
* fix clang-tidy
* fix build
* move class declaration into .cc file
* clang-tidy again
* fix build
* reorganize code and misc cleanups
* clang-format
* fixed a bunch of asan bugs; still some left to debug
* fix asan problem
* make test work on IPv6-only machines
* move LRU list handling into Cache::Entry and add lock annotations
* use preincrement instead of postincrement
* fix deadlock
* add more FIXMEs
* clean up backoff timer logic
* fix build from merge
* clang-format
* fix include path to work on podspec builds
* clean up picker logic
* clang-format
* update rls.proto
* populate reason and stale_header_data in RLS request
* improve logging and fix some cache size type issues
* clang-format
* add some TODOs about a better way to expose channel creds to LB policies
* centralize handling of channel's default authority
* clang-format
* fix backup poller bug
* fix handling of call creds and authority on RLS channel
* remove unused params
* clang-format
* use two-phase update for child policies to avoid deadlocks
* fix portability issue
* minor cleanups
* update for change in grpc_error_get_status() API
* change test to store RLS requests and responses in proto form
* clang-format
* account for test slowdown factor in timeouts
* fix tsan failure and channelz linkage
* fix RLS authority death test
* fix test to not try to connect to target that should not work
* remove illegal term
* cacheSizeBytes is a required field
* add missing BUILD deps from merge
* add tests for connectivity state reporting and fix bugs found
* fix BUILD sanity
* buildifier
* fix BUILD package path
* fix bugs related to child policy lifetime and updates
* remove unnecessary srand() call from grpc_init()
* add test for two cache entries with the same target
* update rls.proto
* change RLS config parser test to support GRPC_ERROR_IS_ABSEIL_STATUS
* update upb codegen
* fix include path
* add env var guard
* avoid duplication of CountedService
* generate_projects
* add upper limit for cacheSizeBytes
* fix build
Co-authored-by: Muxi Yan <mxyan@google.com>
Add `grpc::BinderServerCredentials()` and other related functionalities for the server to listen to binder transactions through a phony "binder port".
The APIs are temporarily placed in internal headers until the corresponding gRFC is merged.
* Avoid copy constructor
```
src/core/lib/gpr/log_linux.cc:78:33: error: copying variable of type 'grpc_core::PthreadTlsImpl<long>' invokes deleted constructor
static GPR_THREAD_LOCAL(long) tid = 0;
^ ~
./src/core/lib/gpr/tls.h:64:3: note: 'PthreadTlsImpl' has been explicitly marked deleted here
PthreadTlsImpl(const PthreadTlsImpl&) = delete;
^
1 error generated.
```
* Generalize pthread TLS to any trivial type
Use multiple pthread keys for types larger than a machine word.
Implement generic timer TLS optimization on all platforms.
This PR imports unit tests and end-to-end tests of binder transport from the internal repository. No further changes will be made to internal repository.
* Add construct/destruct helper functions
Will be used in upcoming promise implementation
* Poll type for promises library
* Library to talk about things that look like promises if you squint
* Helper code for promises to deal with status types generically
* Library to talk about things that make promises
* build
* Join combinator for promise library
* Changes to sync required for promise activities
* sanitized
* Automated change: Fix sanity tests
* Automated change: Fix sanity tests
* Update basic_join.h
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Poll type for promises library
* Library to talk about things that look like promises if you squint
* Library to talk about things that make promises
* Promises loop construct
* build
* Changes to sync required for promise activities
* sanitized
* add comments
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Poll type for promises library
* Library to talk about things that look like promises if you squint
* Library to talk about things that make promises
* Promises if construct
* build
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Poll type for promises library
* Library to talk about things that look like promises if you squint
* Promise helpers, and basic type erasure
* Promise map operator - change return type via a function
* build
* Changes to sync required for promise activities
* sanitized
* Automated change: Fix sanity tests
* fixes
* fixes
* Automated change: Fix sanity tests
* fix build
* review feedback
* Automated change: Fix sanity tests
* comment
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Poll type for promises library
* Library to talk about things that look like promises if you squint
* Library to talk about things that make promises
* build
* Changes to sync required for promise activities
* sanitized
* remove bad comment
* possible windows fix?
* fix
* ugh
* comment fix
* fix build
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Poll type for promises library
* Library to talk about things that look like promises if you squint
* Promise helpers, and basic type erasure
* build
* Changes to sync required for promise activities
* sanitized
* Automated change: Fix sanity tests
* suppressions
* try to fix windows failure
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Limit initial window size increases/decreases by flow control windows of active streams
* Limit stream flow control window updates to maximum allowable
* Reviewer comments
* Alternative way
* Clean-up
* Add tests
* Remove unnecessary cq_verifier
* Generate projects
* Initialize recv_message
* Get around compilation issue
* Test size large
A simple bitset type -- to replace `std::bitset` usage in #26698 and #26254
`std::bitset` uses at least 64 bits even to store two bits, and the usages I'm looking at would benefit from having something smaller in those circumstances
* Add folder for bindertransport and its smoke test
We will import the sources in the next few pull requests. Main purpose
of this commit is to create the folder and add dummy smoke test to make
sure nothing breaks
* regenerate projects.
* match/overload abstraction
* update projects
* match should really not accept mutable args
* typo
* tests
* usage comment
* mutable version
* build stuff
* clang-format
* add an escape hatch to avoid needing port_platform.h in files that do not need port_platform.h
* unused args
* Make it possible for a test to not depend on gpr
* add tests
* compile fix
* sepelling
This code adds an iomgr implementation that's backed by an EventEngine. This uses the EventEngine API alone, and separate work will introduce an EventEngine prototype to plug into it.
See also drfloob#1: @nicolasnoble has a pull request against this branch, implementing the libuv-based EventEngine. One goal here is to implement the iomgr code such that it can be merged independently without affecting normal builds.
This implementation can be built using bazel build --cxxopt='-DGRPC_USE_EVENT_ENGINE' :all
Some shortcuts are being taken to get a working, testable version of the engine. EventEngines are not pluggable, for example.
* Create client_context_test_peer.h
mock-up class for ClientContext
* Add ClientContextTestPeer as a friend for test
* Update client_context_test_peer.h
* Update client_context_test_peer.h
* unit test for client_context_test_peer.h
* unit test for client_context_test_peer.h
* fix for tools/buildgen/generate_projects.sh
* Revert "fix for tools/buildgen/generate_projects.sh"
This reverts commit b007aa8c2b.
* re-fix for tools/buildgen/generate_projects.sh
* Remove an empty line
Co-authored-by: Vijay Pai <vpai@google.com>
* Implement FilterChainMatch logic
* Add tests for transport protocol too
* Tests for duplicate NACKing
* Introduce ConnectionManager as an interface for config fetchers
* Do not parameterize IncrementIfNonZero
* Some formatting
* Reviewer comments
* Add filter chain match information for duplicate match error
* Reviewer comments
* Some cleanup
* Reviewer comments
* Reviewer comments
* Reviewer comments
* Clang-tidy
This caused problems internally. The interface deletion needs a cherrypick, but I will do an overnight test first, before I reintroduce. CC @nicolasnoble
* Implement C++ Admin Interface API
* Address reviewer's requests
* Remove static asserts for raw pointers
* Make sanity tests happy
* Windows: pacify conflict between ifndef and macros
* Disable admin services test on iOS
* Make iOS happy by:
* Letting grpcpp_admin conditionally depend on grpcpp_csds
* Fix an unexpected side-effect of dependency update
Added logging for unsupported attributes
fixed make issues by moving grpc_authorization_engine under grpc
changed inet_aton to inet_pton
fixed issue leftover from initial implementation of evalargs
Merge remote-tracking branch 'upstream/master' into celeval_constructor
Iterate through policies and store CEL condition
git push origin ce Merge remote-tracking branch 'upstream/master' into celeval_constructor
added class and constructor comments for CelEvaluationEngine
CelEvaluationEngine class and constructor
Added cel_evaluation_engine to build
ran clang_tidy_code.sh
updated BUILD with cel_evaluation_engine
added rbac.upb.h/cc to BUILD
Rewrote cel_evaluation_engine code with upb conventions
removed unnecessary deps for cel_evaluation_engine in BUILD
Cel_evaluation_engine uses arenas to manage condition pointers
Used upb::Arena instead of upb_arena*, added temp_arena to save memory
Added action_allow_ member variable
Added fullstops to comments
initialize action_allow_
changed variable names to be more clear
removed unnecessary headers
correctly initialize the action_allow_ variable
changed constructor to use an initializer list
ran clang_tidy and clang_format scripts
added cel_engine_test
resolved merge conflicts and rewrote cel engine constructor to use new map api
changed variables to be more clear and made them const
changed syntax.upb.h so the build succeeds
changed syntax.upb.h so the build succeeds
changed version of upb in bazel_dpes
regenerated upb files with new upb version
added TODO regarding two-policy design
modified CelEvaluationEngine and tests to use two rbac policies
Made test messages consistent and added a case with too many policies
changed name from cel_engine to authorization_engine
fixed merge issues and updated authorization engine to v3 rbac
made constructor public and added namespace grpc_core
Alessio Buraggina