Commit Graph

4326 Commits

Author SHA1 Message Date
AJ Heller 1910986819
[build] Fix clang warnings in Windows code (#32448)
This fixes the problems identified while building with clang-cl on
Windows, with build arguments `/std:c++14 /W4`

Passes internal checks: cl/511562057

----

We can't yet enable a clang-cl build as part of our continuous
integration tests due to a few issues:

protobuf fails an `unused-parameter` warning check in v4.21 (the current
pinned version) on Windows. The upgrade to v4.22 is evidently painful
and in progress. Without maintaining a patch against protobuf, or
disabling warnings-as-errors somehow for the protobuf code alone, we'll
need to upgrade our dependency before we can automate the clang-cl build
for Windows.

Next, our Windows CI environment does not have clang installed. There
has been some work over the past year to create custom kokoro images,
but that work has apparently stalled after trading hands a few times.
Using our current images, installing clang every time we run the job may
be our best bet (likely from precompiled binaries that we host
ourselves), but it will eat up more CI resources.

Finally, some of the default build configurations are incorrect for
clang-cl. For example `-Wall` in clang-cl translates roughly to
`-Weverything` in clang linux, whereas `-W4` in clang-cl translates more
closely to `-Wall -Wextra`. This configuration in the gRPC bazel build
is not currently platform-specific, it will need to be updated.
Similarly, `-std=c++14` is an unknown argument on Windows (should be
`/std:c++14`), and should not be in the bazelrc. This will likely need
the same platform-specific support.
2023-02-23 16:33:34 -08:00
AJ Heller 070ce3630a
[EventEngine] Revert "Revert "EventEngine::RunAt: C++ Alarm (#30024)" (#30147)" (#32461)
This reverts commit 0fc0384b5a.

Major changes: this code calls `GetDefaultEventEngine` once on Alarm
init instead of 7 times throughout.

I will run benchmarks to ensure b/237283941 is not reproduced.


<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->

---------

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2023-02-23 10:05:30 -08:00
Vignesh Babu 8fdc82f33d
[EventEngine] Update Endpoint API to return true if read/write succeeds immediately (#32370)
Requires cherrypick
2023-02-23 09:51:21 -08:00
Yash Tibrewal bf23bb2fa7
Gcp Observability Logging: Batching and Graceful close (#32436)
This PR adds batching support for GCP Observability logging. So instead
of the naive creating a new RPC to cloud logging for each logging event,
we now batch the log events to meet one of the following requirements -
* Batch size of 1000
* Batch memory consumption of 1MB
* A timeout period of 1sec after which we flush the accumulated batch
irrespective of the size.

There can also be cases where for some reason the RPCs fail or the batch
just accumulates to a very large size(100000 entries or 10MB in size).
In such cases, we just log the events with gpr_log instead of just
continuing to accumulate.

Additionally, `GcpObservabilityClose()` has been added to gracefully
shut off logging where we block till all the currently logged events are
flushed. (We might be able to gracefully shut off stats and tracing in
the future too.)

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-02-22 22:32:44 -08:00
Yash Tibrewal d93d0022cd
GCP Observability: Fix constant labels support for stats when tracing is disabled (#32451)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-02-22 14:49:55 -08:00
Mark D. Roth 50e6af14ff
Bump dev version to 1.54.0-dev (#32426) 2023-02-21 12:03:19 -08:00
AJ Heller ffe3968d0b
[EventEngine] Add advice against blocking work in callbacks (#32397)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-02-15 14:00:55 -08:00
Yousuk Seung b98f527260
Revert "Revert "Revert "Revert "server: introduce ServerMetricRecorde… (#32301)
* Revert "Revert "Revert "Revert "server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)" (#32272)" (#32279)" (#32293)"

This reverts commit 1f960697c5.

* Do not create CallMetricRecorder if call is null.
2023-02-06 12:00:39 -08:00
Craig Tiller 1f960697c5
Revert "Revert "Revert "server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)" (#32272)" (#32279)" (#32293)
This reverts commit 4475e74c6a.
2023-02-03 13:18:23 -08:00
Yousuk Seung 4475e74c6a
Revert "Revert "server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)" (#32272)" (#32279)
* Revert "Revert "server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)" (#32272)"

This reverts commit deb1e25543.

* Fix by caching call metric recording stuff in async request

PR #32106 caused msan errors in some tests while de-referencing the
server object where async calls are active after the server is
destroyed. Instead cache the ServerMetricRecorder pointer.

* copyright headers fixed

* clang fixes.
2023-02-03 11:49:32 -08:00
Yijie Ma 08555c8723
Add enforcement for cpp-style comment license in check_copyright.py (#32273)
* enforce cpp style comment copyright license in check_copyright.py

also add functionality to fix(replace) c-style comment with cpp style
comment

* fix tools/codegen/core/gen_stats_data.py

* review
2023-02-02 13:00:20 -08:00
Xuan Wang deb1e25543
Revert "server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)" (#32272)
This reverts commit c7f641da0d.
2023-02-02 09:58:29 -08:00
Yousuk Seung c7f641da0d
server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)
* backend metric sampling

* Comments addressed.

* More comments addressed.

* Pushing changes left behind locally.

* Removed empty lines

* Update OrcaService to use ServerMetricRecorder (no named metrics yet)

* Comments addressed.

* More comments addressed

* More comments addressed.

* Comments fixed

* Comments addressed.

* Test fixed

* make seq returned always up-to-date

* skip atomic load when not cached

* Fixed ABSL_GUARDED_BY

* Comments addressed except client_lb_end2end_test

* test updated

* Comments addressed

* BUILD fix.

* BackendMetricDataState moved to a separate header

* comments addressed

* Fixed clang and buildifier errors

* More sanity check errors fixed.

* Fixed xds tests

* Ran generate_projects.sh

* Comments addressed

* comments addressed.

* generate project

* Build fixed

* generate project

* sanity check errors fixed

* test fixed

* Backup poller period override moved to main()

* Also move cfstream override

* Clang fixes, sanitize

* generate_projects.sh

* portable print format fix

* Removed outdated comment
2023-02-01 14:54:57 -08:00
AJ Heller d14020d4e2
[EventEngine] Add invalid handle types to the public API (#32202)
* [EventEngine] Add invalid handle types to the public API

* Automated change: Fix sanity tests

* add definition for static constexpr members

* sanitize

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2023-01-25 17:31:34 -08:00
AJ Heller 6a92df7237
[fixup] EventEngine Nomenclature (#32195) 2023-01-25 09:31:46 -08:00
Craig Tiller 7b02b7c253
[stats] Add experiment to use transport reported request latency whenever available (#32168)
* [stats] Add experiment to use transport reported request latency whenever available

* redux: add a new metric

* review-feedback

* Automated change: Fix sanity tests

* fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-01-24 08:09:59 -08:00
Vignesh Babu 8cf04e9a54
[EventEngine] Modify iomgr to allow creation of event engine clients and client side endpoints (#31661)
* [WIP] EventEngine iomgr endpoint shims

* [WIP] EventEngine::Endpoint iomgr shims for the PosixEventEngine

* Util functions to help with posix engine listener implementation

* sanity

* update comments in posix_engine_listener_utils.h

* review comments

* iwyu

* revert prev commit

* iwyu

* update build

* update

* regenerate projects

* regenerate projects

* minor fixes

* update BUILD

* sanity

* update build

* regenerate projects

* fix unused parameter

* sanity

* update

* sanity

* regenerate_projects

* remove unused variable

* start

* update

* regenerate_projects

* sanity

* update

* fixes

* update

* regenerate_projects

* update

* fix sanity and msan failure

* more fixes

* build failure

* update

* fix

* sanity

* fixes

* update

* regenerate projects

* fix sanity

* review comments

* An EventEngine subclass to be implemented by all posix based event engines

* sanity

* comments

* update

* review comments

* re-word

* fix

* update

* review comments

* regenerate projects

* syntax fix

* add lock free event benchmark

* releasable mutex lock

* fix build isue

* update

* start

* regenerate projects

* update

* fix

* windows build

* update

* windows portability issue

* update

* update

* update

* update

* format

* update

* update

* update

* update

* update

* fix sanity

* regenerate projects

* update

* iwyu

* Fix resolved address length related bugs in tcp_socket_utils and listener_utils

* iwyu

* cleanup src/core/lib/event_engine/tcp_socket_utils.cc

* iwyu

* fix

* regenerate projects

* fix sanity

* re-write endpoint shim

* more re-write

* cleanup

* update

* review comments

* build issue

* more build issue fixes plus adding event_engine_trace

* even more build issue fixes

* iwyu

* add static_cast

* update

* remove redundant code

* update

* deduplicate

* iwyu

* Fix review comments and regenerate_projects

* sanity

* review comments

* fix include guards

Co-authored-by: AJ Heller <hork@google.com>
2023-01-20 19:47:14 -08:00
Yijie Ma c5b8fd681f
Add an additional < marker in comment blocks after the documented functions (#32114)
* Add an additional < marker in the comment block for documentation block
after the member

* review
2023-01-18 16:55:51 -08:00
Yash Tibrewal 05491fb2f6
Gcp Observabiliy : Add constant labels support for stats and tracing (#32128)
* Gcp Observabiliy : Add constant labels support for stats and tracing

* Register GCP Observability labels

* sanity

* Fix build for CI

* TEST_TAG_KEY fix

* Remove duplicitous constant label setting
2023-01-18 14:32:36 -08:00
AJ Heller 8ac043638c
[EventEngine] Add ResolvedAddress default constructor (#32142)
* [EventEngine] Add ResolvedAddress default constructor

* alternative
2023-01-18 12:22:38 -08:00
Jan Tattermusch 7fd808f6f9
Bump version to 1.53.0-dev (on master branch) (#32086)
* bump version to 1.53.0-dev

* regenerate projects
2023-01-18 09:41:33 +01:00
Yousuk Seung 6b31b8d346
Add MakeChannelArgumentOption() for void* (#32096) 2023-01-13 17:15:30 -08:00
Craig Tiller 67f364e23e
[cleanup] Eliminate usage of GRPC_ASSERT(false...); (#31757)
* crash function

* progress

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fixes

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

* fix

* fix

* use cpp attr

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* fix

* fix

* add exclusion

* fix

* typo

* fix

* fmt

* Update tcp_socket_utils.cc

* Automated change: Fix sanity tests

* fix

* revert php changes

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-01-11 08:50:32 -08:00
Yash Tibrewal 75f2a2c1c6
GCP Observability: Client attempt top latency (#32064) 2023-01-10 13:40:33 -08:00
Mark D. Roth b0d7131ab3
WRR: plumb qps in backend metric data (#31903)
* plumb qps in backend metric data

* pull in updated orca proto

* gen_upb_api

* Automated change: Fix sanity tests

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2023-01-09 10:07:58 -08:00
AJ Heller 5db6de8634
[༺ EventEngine ༻] Specify requirements for Run* immediate execution (#32028)
* [༺ EventEngine༻ ] Specify requirements for Run* immediate execution

Also adds test suite tests that may catch non-conforming implementations
(Run called ~3200 times in case it's non-deterministic).

* fix

* verbiage; longer timeout

* fix int type comparison warning

* rm illegal term

* Automated change: Fix sanity tests

* disable tests

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2023-01-05 16:26:00 -08:00
Vignesh Babu 8bd2b9af52
Fixing some bugs in posix event engine (#31785)
* update

* regenerate projects

* fix sanity

* review comments

* fix

* add lock free event benchmark

* releasable mutex lock

* fix build isue

* update

* fix sanity

* regenerate projects

* iwyu

* Fix resolved address length related bugs in tcp_socket_utils and listener_utils

* iwyu

* cleanup src/core/lib/event_engine/tcp_socket_utils.cc

* update endpoint to use SliceCast into MutableSlice

* fix sanity

* review comments

* review comments

* regenerate projects

* fix

* rm unused variable

* fix
2023-01-03 17:52:46 -08:00
Vignesh Babu 4f9ba0f49b
Expand SliceCast to support casts to MutableSlice type (#31996)
* Expand SliceCast to support casts to MutableSlice type

* update
2022-12-30 11:36:12 -08:00
Yijie Ma f99b8b5bc4
Convert c-style comments to C++-style comments (#31923)
* baseline

* fix clang-tidy

* manually revert these files

* manually fixup at eof

* revert 2 more files

* change check_deprecated_grpc++.py

* change end2end_defs.include template

* fix check_include_guards.py

* untrack tools/distrib/python/convert_cstyle_comments_to_cpp.py

not yet ready to be submitted

* fix

yapf check_include_guards.py
remove a space...

* fix version.cc.template

* fix version_info.h.template
2022-12-22 23:01:53 -08:00
Craig Tiller 5664d6ee6b
[slice] Mutable slice cast (#31959)
* [slice] Mutable slice cast

* fix
2022-12-22 12:45:25 -08:00
Craig Tiller 72c296a3c2
[event_engine] Add SliceCast (#31831)
* [event_engine] Add SliceCast

* Automated change: Fix sanity tests

* windows-fix

* comments

* comments

* comments

* build-fix

* fix

* Update port_platform.h

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-12-20 19:46:27 -08:00
Yash Tibrewal 3683ff924b
GCP Observability: Add server latency metric (#31900) 2022-12-16 09:32:56 -08:00
Cheng-Yu Chung 1b05576aa3
[cleanups] Clean up `GrpcLibraryInitializer` (#31894)
* Clean up `GrpcLibraryInitializer`

* Update

* Automated change: Fix sanity tests
2022-12-17 01:08:35 +08:00
Yash Tibrewal 6a97f492ff
Add method to add annotations to call tracer (#31901) 2022-12-16 02:28:59 +05:30
Yash Tibrewal 5100b33e0e
GCP Observability: Add payload metrics (#31899) 2022-12-15 11:07:09 -08:00
Yash Tibrewal a0f00f0956
GCP Observability: Fix view names for started and completed rpcs (#31898) 2022-12-15 23:54:12 +05:30
Cheng-Yu Chung b72da316bb
[cleanups] Remove `include/grpc/impl/codegen/slice.h` (#31871)
* Remove `include/grpc/impl/codegen/slice.h`

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests
2022-12-15 14:36:04 +08:00
Cheng-Yu Chung b887534111
[cleanups] Remove `include/grpc/impl/codegen/propagation_bits.h` (#31870)
* Remove `include/grpc/impl/codegen/propagation_bits.h`

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests
2022-12-15 11:08:02 +08:00
Cheng-Yu Chung e28419f1f9
[cleanups] Remove `class CoreCodegen` and `g_core_codegen_interface` (#31875)
* Delete `g_core_codegen_interface`

* Automated change: Fix sanity tests

* Fix

* Update
2022-12-14 23:47:13 +08:00
Cheng-Yu Chung ac365cadfb
Remove `include/grpc/impl/codegen/port_platform.h` (#31778) 2022-12-14 10:46:33 +08:00
Cheng-Yu Chung 9a7e77318b
[cleanups] Remove `include/grpc/impl/codegen/grpc_types.h` (#31835)
* Remove `include/grpc/impl/codegen/grpc_types.h`

* Automated change: Fix sanity tests

* Fix typo

* Automated change: Fix sanity tests
2022-12-14 02:29:17 +08:00
Rokya feda5031fa
[BinderTransport] Make `CreateBinderChannel` parameter names consistent (#31852)
* Rename `jni_env` to `jni_env_void`
* Rename `application` to `context`
2022-12-13 00:18:45 -08:00
Cheng-Yu Chung f03304b963
[cleanups] Remove functions in `class CoreCodegen` (#31837)
* Clean up `assert_fail()`

* Clean up `ok()` and `cancel()`

* Automated change: Fix sanity tests
2022-12-13 09:37:48 +08:00
Cheng-Yu Chung 2e9e6439f8
Remove `include/grpc/impl/codegen/connectivity_state.h` (#31728)
* Remove `include/grpc/impl/codegen/connectivity_state.h`

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests
2022-12-12 18:03:33 +08:00
Cheng-Yu Chung 0356ff3a72
Remove `include/grpc/impl/codegen/gpr_types.h` (#31818)
* Remove `include/grpc/impl/codegen/gpr_types.h`

* Automated change: Fix sanity tests
2022-12-10 02:31:44 +08:00
Cheng-Yu Chung c3132358d6
Remove `include/grpc/impl/codegen/status.h` (#31817)
* Remove `include/grpc/impl/codegen/status.h`

* Automated change: Fix sanity tests
2022-12-09 07:48:50 +08:00
Eryu Xia a8ac4e7265
Revert "[ObjC]fix 64 to 32 bit clang conversion warning in src/core/lib (#29877)" (#31843)
This reverts commit eb0a591811.
2022-12-08 15:33:49 -08:00
Hannah Shi eb0a591811
[ObjC]fix 64 to 32 bit clang conversion warning in src/core/lib (#29877) 2022-12-08 14:46:38 -08:00
Mark D. Roth a34f9e170f
ring_hash LB: cap ring size to 4096 with channel arg to override (#31692)
* ring_hash LB: cap ring size to 4096 with channel arg to override

* change default max_ring_size value to 4096 in LB policy config
2022-12-07 09:33:43 -08:00
Cheng-Yu Chung 4e90f43a75
Remove functions in `class CoreCodegen` (#31796)
* Update include

* Clean up `grpc_empty_slice()`

* Clean up `grpc_slice_malloc()`

* Clean up `grpc_slice_unref()`

* Clean up `grpc_slice_ref()`

* Clean up `grpc_slice_split_tail()`

* Clean up `grpc_slice_split_head()`

* Clean up `grpc_slice_sub()`

* Clean up `grpc_slice_buffer_add()`

* Clean up `grpc_slice_buffer_add_indexed()`

* Clean up `grpc_slice_buffer_pop()`

* Clean up `grpc_slice_from_static_buffer()`

* Clean up `grpc_slice_from_copied_buffer()`

* Clean up `grpc_metadata_array_init()`

* Clean up `grpc_metadata_array_destroy()`

* Clean up `gpr_inf_future()`

* Clean up `gpr_time_0()`
2022-12-07 18:55:47 +08:00