1c885232e0
Reland te:trailers metadata change with fixes ( #27785 )
...
* Revert "Revert "Reland te: trailers metadata stuff (#27781 )" (#27784 )"
This reverts commit 33fff40174
2021-10-20 14:50:33 -07:00
33fff40174
Revert "Reland te: trailers metadata stuff ( #27781 )" ( #27784 )
...
This reverts commit f5ab108ab7
2021-10-20 11:21:53 -07:00
f5ab108ab7
Reland te: trailers metadata stuff ( #27781 )
...
* Revert "Revert "Add metadata specialization for te: trailers (#27714 )" (#27779 )"
This reverts commit 2039cec4b1
2021-10-20 08:29:58 -07:00
2039cec4b1
Revert "Add metadata specialization for te: trailers ( #27714 )" ( #27779 )
...
This reverts commit 7cded136b7
2021-10-19 16:42:41 -07:00
7cded136b7
Add metadata specialization for te: trailers ( #27714 )
...
* Add metadata specialization for te: trailers
* remove the evidence
* update microbenchmarks
* ugh
* cleanup copy code
* fix asan detected leak
* Automated change: Fix sanity tests
* append
* binder
* fix-cronet
* inproc fixes
* Automated change: Fix sanity tests
* convert cronet encode path
* fixes
* fixes
* review feedback
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-10-19 16:23:22 -07:00
632cc4cc36
Make hpack tracing available in opt builds also ( #27728 )
2021-10-15 08:47:56 -07:00
1df153f46e
Revert "Revert "Move metadata setting into HPackParser ( #27545 )" ( #27686 )" ( #27688 )
...
This reverts commit 0e6ced8800
2021-10-13 08:44:57 -07:00
0e6ced8800
Revert "Move metadata setting into HPackParser ( #27545 )" ( #27686 )
...
This reverts commit f57a1f7d8c
2021-10-12 08:37:00 -07:00
f57a1f7d8c
Move metadata setting into HPackParser ( #27545 )
...
* forward work to get append, limit check into parser
* further cleanup
* Remove chttp2_incoming_metadata_buffer
* Automated change: Fix sanity tests
* test fixes
* fix cronet
* wip
* first pass mementofication of parsing - libgrpc compiles
* further progress
* fixes
* fixes
* fix leak
* Automated change: Fix sanity tests
* fix leak
* fixes
* x
* improve fuzzer
* init ordering fix
* Update hpack_parser_table.h
* Update hpack_parser.cc
* Update hpack_parser_test.cc
* Automated change: Fix sanity tests
* Automated change: Fix sanity tests
* i mustache myself why
* fix use after free
* Automated change: Fix sanity tests
* fixes
* eliminate second pass metadata - this is an illegal frame and fails the new parser
* handle mementos with size > sizeof(intptr_t)
* fix
* add parsed metadata header, test
* Automated change: Fix sanity tests
* fix includes
* phase1
* phase2
* Update parsed_metadata.h
* fix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-10-11 20:02:29 -07:00
b2942f601c
Modify hpack benchmark scenarios ( #27567 )
...
It used to be that we could run the hpack deserializer into an arbitrary
sink - that's about to change and the hpack deserializer will buffer
directly into a map. That being the case, it's really hard to compare
performance of the current and the new system. With this PR, I'm
updating the relevant scenario to be more similar to the eventual
scenario that we plan, so that as the new work finishes up, we can get a
clearer idea of what changes we're really making.
2021-10-04 13:14:43 -07:00
91a2db9bc2
Make deadline the first metadata trait ( #27468 )
2021-09-28 10:09:04 -07:00
9872da79a1
Eliminate grpc_metadata_batch_{init,destroy,move} ( #27349 )
...
* Eliminate grpc_metadata_batch_{init,destroy,move}
* fixes
* fixes
* fixes
* fixes
* portability fixes
* fixes
* fix
* fix
* fix
* be explicit about being explicit
2021-09-20 17:25:26 -07:00
fd233193ec
Encapsulate metadata aggregates ( #27262 )
...
* Encapsulate metadata aggregates
* Automated change: Fix sanity tests
* add const
* Automated change: Fix sanity tests
* add a comment about an awful api
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-14 09:45:38 -07:00
ea389c00c2
Adjust include order per style guide ( #27175 )
...
Introduce clang-format configuration to sort includes closer to our rules.
2021-09-08 12:14:44 -07:00
dc701787e2
Convert HPACK encoder to C++ ( #27226 )
...
* Rebuild HPACK encoder table as C++
* move comment
* incguards
* build
* Automated change: Fix sanity tests
* c++ initialization ftw
* Automated change: Fix sanity tests
* Add missing header
* Add missing header
* Begin converting HPACK encoder to c++
* First pass conversion to c++
* fixes
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-07 14:28:06 -07:00
bbea27de46
Reland HPACK parsing changes ( #26997 )
...
* Revert "Revert "HPACK Table --> C++ (#26851 )" (#26995 )"
This reverts commit 840bcce9c4
2021-08-13 10:57:42 -07:00
840bcce9c4
Revert "HPACK Table --> C++ ( #26851 )" ( #26995 )
...
This reverts commit 83bcb0cf2e
2021-08-12 11:20:19 -07:00
83bcb0cf2e
HPACK Table --> C++ ( #26851 )
...
* Buffer HPACK parsing until the end of a header boundary
HTTP2 headers are sent in (potentially) many frames, but all must be
sent sequentially with no traffic intervening.
This was not clear when I wrote the HPACK parser, and still indeed quite
contentious on the HTTP2 mailing lists.
Now that matter is well settled (years ago!) take advantage of the fact
by delaying parsing until all bytes are available.
A future change will leverage this to avoid having to store and verify
partial parse state, completely eliminating indirect calls within the
parser.
* maybe fixes
* xx
* fix boundary detection
* clang-format
* Revert "xx"
This reverts commit 258d712ed3#26700 )"
This reverts commit 8bab3e4bf4
2021-08-09 16:07:41 -07:00
613b90ba93
Revert "Buffer HPACK parsing until the end of a header boundary ( #26700 )" ( #26825 )
...
This reverts commit 8bab3e4bf4
2021-07-29 11:22:47 -07:00
8bab3e4bf4
Buffer HPACK parsing until the end of a header boundary ( #26700 )
...
HTTP2 headers are sent in (potentially) many frames, but all must be
sent sequentially with no traffic intervening.
This was not clear when I wrote the HPACK parser, and still indeed quite
contentious on the HTTP2 mailing lists.
Now that matter is well settled (years ago!) take advantage of the fact
by delaying parsing until all bytes are available.
A future change will leverage this to avoid having to store and verify
partial parse state, completely eliminating indirect calls within the
parser.
2021-07-28 15:36:51 -07:00
d3e5803cb2
Move HPACK parser to C++ ( #26689 )
...
This is a fairly low effort migration of the current codebase into a C++ class, instead of free standing C code.
It builds upon #26657 as a necessary first step.
I've tried to minimize any changes to semantics or logic in this change, except where required to get a minimal amount of encapsulation - which is the major aim of this change.
A future change in this series will buffer slices until all HPACK headers are in memory for a stream prior to decoding -- it's important to have an encapsulated API to the parser before doing so however (hence this CL).
The next change after that will be an almost complete rewrite of the parsing functionality -- since we'll have the total set of header bytes, we'll no longer need to support suspending decoding at arbitrary points. This will allow us to move to a simple recursive descent parser, eliminate a bunch of indirection in this code, and end up in a much more malleable place for when we start doing metadata API changes.
(we likely also end up with some good performance wins!)
2021-07-15 13:51:23 -07:00
ca945a58e9
Introduced grpc_error_handle ( #25902 )
...
- Define grpc_error_handle
- Replace grpc_error* with grpc_error_handle
2021-04-22 12:00:28 -07:00
62ac3f075a
Added call to grpc::testing::TestEnvironment in tests
2020-07-01 11:37:57 -07:00
3a519a0b64
Replaced grpc_core New & Delete with C++ new & delete
2019-11-06 17:52:26 -08:00
406b70629a
Remove unused parameter warning (17 of 20)
2019-10-17 08:52:00 -07:00
0b06676c9e
hpack encoder optimizations.
...
Removed some cycles and branches from hpack_enc for CH2.
Specifically:
1. Pushed certain metadata key/value length checks to
prepare_application_metadata() in src/core/lib/surface/call.cc.
This means that rather than check all key/val lengths for all metadata, we only
do so for custom added user metadata. Inside CH2, we change the length checks to
debug checks so we can catch if core/filter metadata fails to pass the check.
2. Changed various asserts to debug asserts when able.
3. Refactored some of the header emission code to remove duplicated code.
4. Un-inlined some logging methods.
This results in somewhat faster hpack_encoder performance:
BM_HpackEncoderInitDestroy
222ns ± 0% 221ns ± 0% -0.29% (p=0.000 n=34+34)
BM_HpackEncoderEncodeDeadline
[framing_bytes/iter:9 header_bytes/iter:6 ] 135ns ± 1%
124ns ± 0% -8.05% (p=0.000 n=39+38)
BM_HpackEncoderEncodeHeader<EmptyBatch>/0/16384
[framing_bytes/iter:9 header_bytes/iter:0 ] 34.2ns ± 0%
34.2ns ± 0% -0.01% (p=0.014 n=34+38)
BM_HpackEncoderEncodeHeader<EmptyBatch>/1/16384
[framing_bytes/iter:9 header_bytes/iter:0 ] 34.2ns ± 0%
34.2ns ± 0% -0.04% (p=0.004 n=34+37)
BM_HpackEncoderEncodeHeader<SingleStaticElem>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.5ns ± 0%
45.9ns ± 0% -3.28% (p=0.000 n=28+38)
BM_HpackEncoderEncodeHeader<SingleInternedKeyElem>/0/16384
[framing_bytes/iter:9 header_bytes/iter:6 ] 77.0ns ± 1%
68.3ns ± 1% -11.33% (p=0.000 n=39+40)
BM_HpackEncoderEncodeHeader<SingleInternedElem>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.7ns ± 1%
45.5ns ± 0% -4.63% (p=0.000 n=39+33)
BM_HpackEncoderEncodeHeader<SingleInternedBinaryElem<1, false>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.2ns ± 0%
45.3ns ± 0% -3.96% (p=0.000 n=33+34)
BM_HpackEncoderEncodeHeader<SingleInternedBinaryElem<3, false>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.7ns ± 0%
45.6ns ± 0% -4.54% (p=0.000 n=38+40)
BM_HpackEncoderEncodeHeader<SingleInternedBinaryElem<10, false>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.7ns ± 0%
45.5ns ± 0% -4.63% (p=0.000 n=39+32)
BM_HpackEncoderEncodeHeader<SingleInternedBinaryElem<31, false>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.8ns ± 0%
45.6ns ± 1% -4.59% (p=0.000 n=38+39)
BM_HpackEncoderEncodeHeader<SingleInternedBinaryElem<100, false>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.8ns ± 0%
45.5ns ± 0% -4.64% (p=0.000 n=39+36)
BM_HpackEncoderEncodeHeader<SingleInternedBinaryElem<1, true>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.3ns ± 0%
45.3ns ± 0% -4.09% (p=0.000 n=38+36)
BM_HpackEncoderEncodeHeader<SingleInternedBinaryElem<3, true>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.8ns ± 1%
45.6ns ± 0% -4.71% (p=0.000 n=37+40)
BM_HpackEncoderEncodeHeader<SingleInternedBinaryElem<10, true>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.7ns ± 0%
45.5ns ± 0% -4.66% (p=0.000 n=39+32)
BM_HpackEncoderEncodeHeader<SingleInternedBinaryElem<31, true>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.8ns ± 1%
45.6ns ± 1% -4.62% (p=0.000 n=37+39)
BM_HpackEncoderEncodeHeader<SingleInternedBinaryElem<100, true>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.7ns ± 0%
45.5ns ± 0% -4.67% (p=0.000 n=38+32)
BM_HpackEncoderEncodeHeader<SingleNonInternedElem>/0/16384
[framing_bytes/iter:9 header_bytes/iter:9 ] 80.5ns ± 1%
74.7ns ± 0% -7.16% (p=0.000 n=38+35)
BM_HpackEncoderEncodeHeader<SingleNonInternedBinaryElem<1, false>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:12 ] 105ns ± 1%
99ns ± 0% -5.91% (p=0.000 n=38+34)
BM_HpackEncoderEncodeHeader<SingleNonInternedBinaryElem<3, false>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:14 ] 111ns ± 1%
106ns ± 1% -4.86% (p=0.020 n=39+2)
BM_HpackEncoderEncodeHeader<SingleNonInternedBinaryElem<10, false>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:23 ] 135ns ± 0%
130ns ± 0% -3.45% (p=0.020 n=35+2)
BM_HpackEncoderEncodeHeader<SingleNonInternedBinaryElem<31, false>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:46 ] 225ns ± 1%
223ns ± 0% -0.91% (p=0.003 n=37+2)
BM_HpackEncoderEncodeHeader<SingleNonInternedBinaryElem<100, false>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:120 ] 467ns ± 0%
472ns ± 0% +1.09% (p=0.003 n=38+2)
BM_HpackEncoderEncodeHeader<SingleNonInternedBinaryElem<1, true>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:12 ] 81.6ns ± 1%
74.8ns ± 0% -8.40% (p=0.000 n=37+33)
BM_HpackEncoderEncodeHeader<SingleNonInternedBinaryElem<3, true>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:14 ] 82.0ns ± 1%
74.8ns ± 0% -8.80% (p=0.000 n=37+32)
BM_HpackEncoderEncodeHeader<SingleNonInternedBinaryElem<10, true>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:21 ] 82.1ns ± 1%
74.9ns ± 0% -8.86% (p=0.000 n=35+34)
BM_HpackEncoderEncodeHeader<SingleNonInternedBinaryElem<31, true>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:42 ] 97.6ns ± 2%
91.8ns ± 0% -5.95% (p=0.000 n=35+27)
BM_HpackEncoderEncodeHeader<SingleNonInternedBinaryElem<100, true>>/0/16384
[framing_bytes/iter:9 header_bytes/iter:111 ] 97.2ns ± 1%
91.2ns ± 2% -6.19% (p=0.000 n=37+38)
BM_HpackEncoderEncodeHeader<SingleNonInternedElem>/0/1
[framing_bytes/iter:54 header_bytes/iter:9 ] 230ns ± 0%
221ns ± 0% -3.91% (p=0.000 n=38+37)
BM_HpackEncoderEncodeHeader<MoreRepresentativeClientInitialMetadata>/0/16384
[framing_bytes/iter:9 header_bytes/iter:16 ] 206ns ± 2%
170ns ± 1% -17.51% (p=0.000 n=39+39)
BM_HpackEncoderEncodeHeader<RepresentativeServerInitialMetadata>/0/16384
[framing_bytes/iter:9 header_bytes/iter:3 ] 66.4ns ± 2%
62.5ns ± 1% -5.85% (p=0.000 n=34+39)
BM_HpackEncoderEncodeHeader<RepresentativeServerTrailingMetadata>/1/16384
[framing_bytes/iter:9 header_bytes/iter:1 ] 47.5ns ± 0%
45.9ns ± 1% -3.29% (p=0.000 n=26+38)
2019-09-06 13:43:29 -07:00
9bb1247bdb
Merge pull request #16327 from vjpai/keep_running
...
C++ microbenchmarks: update from KeepRunning to range-based for
2019-09-05 11:07:53 -07:00
1077b3435c
Use range-based for on state rather than state.KeepRunning when possible
2019-08-29 11:27:00 -07:00
b46e3668d3
s/branch/tail_call/ for CH2 on_hdr().
...
on_hdr() checks if a void-return function pointer is null before jumping to it.
If it is null, it returns an error; else it executes that function and returns
success.
This change converts the void-returning function to one that returns a
grpc_error* and thus saves a branch in on_hdr() (since we're branching once by
following the function pointer anyways, we're effectively coalescing these two
branches).
2019-08-28 13:06:11 -07:00
59564ebd96
Fix warnings to unblock gcc8 support
2019-08-20 16:02:53 -07:00
b1d73a01f1
Removed duplicate static table from hpack table. Removed an or instruction for
...
every usage of static grpc metadata. Inlined hpack table lookups for static
metadata.
This leads to faster hpack parser creation:
BM_HpackParserInitDestroy 5.32µs ± 1% 0.06µs ± 1% -98.91% (p=0.000 n=18+19)
And slightly faster parsing:
BM_HpackParserParseHeader<RepresentativeClientInitialMetadata, OnInitialHeader>
456ns ± 1% 435ns ± 1% -4.74% (p=0.000 n=18+19)
BM_HpackParserParseHeader<MoreRepresentativeClientInitialMetadata,
OnInitialHeader>
1.06µs ± 2% 1.04µs ± 2% -1.82% (p=0.000 n=19+20)
It also yields a slight (0.5 - 1.0 microsecond) reduction in CPU time for
fullstack unary pingpong:
BM_UnaryPingPong<TCP, NoOpMutator, NoOpMutator>/0/512
[polls/iter:3.0001 ] 23.9µs ± 2%
23.0µs ± 1% -3.63% (p=0.002 n=6+6)
BM_UnaryPingPong<TCP, NoOpMutator, NoOpMutator>/0/32768
[polls/iter:3.00015 ] 35.1µs ± 1%
34.2µs ± 1% -2.57% (p=0.036 n=5+3)
BM_UnaryPingPong<MinTCP, NoOpMutator, NoOpMutator>/8/0
[polls/iter:3.00011 ] 21.7µs ± 3%
21.2µs ± 2% -2.44% (p=0.017 n=6+5)
2019-07-15 15:39:31 -07:00
e18ed03c04
Made gRPC inialized after entering main function in microbenchmarks.
2019-05-02 16:58:37 -07:00
8ce42f67b2
Shrink arena size by 40 bytes and add additional
...
alignment options (for cache-alignment).
We shrink by:
1) Removing an unnecessary zone pointer.
2) Replacing gpr_mu (40 bytes when using pthread_mutex_t) with
std::atomic_flag.
We also header-inline the fastpath alloc (ie. when not doing a zone
alloc) and move the malloc() for a zone alloc outside of the mutex
critical zone, which allows us to replace the mutex with a spinlock.
We also cache-align created arenas.
2019-04-24 16:39:03 -07:00
48e4a81b05
Remeve memset(0) from arena allocated memory.
...
Callers are updated to properly initialize the memory.
This behavior can be overridden using GRPC_ARENA_INIT_STRATEGY
environment variable.
2018-11-05 10:12:39 -05:00
4c6e7ce15d
Destroy metadata buffer at end of benchmark loop
2018-09-20 12:56:56 -07:00
d44feec92f
Reassign arena pointer instead of stomping on memory
2018-09-20 09:54:59 -07:00
4b721fbde0
Destroy arena at end of benchmark to not leak memory
2018-09-19 20:51:45 -07:00
29d9489ea9
Increase initial arena size to be more representative of real workload scenario and increase frequency of recreating the arena to avoid oom
2018-09-19 20:46:17 -07:00
91727bd015
Move arena create outside of benchmark, format, and typo fix
2018-09-19 19:13:23 -07:00
967bbcd5d3
Fixing benchmark name and adding a new one
2018-09-13 18:38:01 -07:00
58e0cbf9fb
Enable the performance-* clang-tidy checks
2018-06-14 14:58:09 -04:00
40ec89ff67
Support microbenchmarks internally
2018-04-25 17:41:23 -07:00
2f4161c210
Use stack frame size limits for consistency with internal builds
2018-03-05 16:43:32 -08:00
4d20a66685
Run clang fmt
2018-02-09 09:34:04 -08:00
be82e64b3d
Autofix c casts to c++ casts
2018-02-09 09:16:55 -08:00
8cf1470a51
Revert "Revert "All instances of exec_ctx being passed around in src/core removed""
2017-12-06 09:47:54 -08:00
ad4d2dde00
Revert "All instances of exec_ctx being passed around in src/core removed"
2017-12-06 09:05:05 -08:00
c354269ba7
Remove _ prefixed variable names
2017-11-29 19:07:44 -08:00
177039b2f8
Merge master
2017-11-16 19:31:28 -08:00
6c26b16fe0
Move ExecCtx to grpc_core namespace. Make exec_ctx a private static in ExecCtx and some minor changes
2017-11-14 18:11:22 -08:00
Craig Tiller