Adds experimental fork support to gRPC/Ruby
Works towards https://github.com/grpc/grpc/issues/8798 (see caveats for why this wasn't marked fixed yet)
Works towards https://github.com/grpc/grpc/issues/33578 (see caveats for why this wasn't marked fixed yet)
This leverages existing `pthread_atfork` based C-core support for
forking that python/php use, but there's a bit extra involved mainly
because gRPC/Ruby has additional background threads.
New tests under `src/ruby/end2end` show example usage.
Based on https://github.com/grpc/grpc/pull/33495
Caveats:
- Bidi streams are not yet supported (bidi streams spawn background
threads which are not yet fork safe)
- Servers not supported
- Only linux supported
Remove `src/ruby/ext/grpc/objs/` and strip `src/ruby/ext/grpc/libs/` in
ruby build dir to save some space.
This can reduce the grpc gem size from more than 1G to about 70~80M.
This only affects opt build, dbg build remains the same.
A proper fix maybe use the same makefile to build c-core and the
extension rather than building c-core when generating the extension
makefile.
fixes: https://github.com/grpc/grpc/issues/33412
This PR does the following: for the TLS server credentials, stops
calling `SSL_CTX_set_client_CA_list` by default in
`ssl_transport_security.cc`, and gives users a knob to re-enable calling
this API.
## What does the `SSL_CTX_set_client_CA_list` API do?
When this API is called, a gRPC TLS server sends the following data in
the ServerHello: for each certificate in the server's trust bundle, the
CA name in the certificate.
This API does not change the set of certificates trusted by the server
in any way. Rather, it is just providing a hint to the client about what
client certificate should be sent to the server.
## Why are we removing the use of `SSL_CTX_set_client_CA_list` by
default for the TLS server credentials?
Removing the use of this API by default has 2 benefits:
1. Calling this API makes gRPC TLS unusable for servers with a
sufficiently large trust bundle. Indeed, if the server trust bundle is
too large, then the server will always fail to build the ServerHello.
2. Calling this API is introducing a huge amount of overhead (1000s of
bytes) to each ServerHello, so removing this feature will improve
connection establishment latency for all users of the TLS server
credentials.
Add -weak_framework CoreFoundation to the ruby extension link line on
arm64-darwin, to address "undefined symbols" issues from #33483
This is a variation on #33513 in response to feedback and so I've made
sure @alto-ruby is credited as co-author.
Closes#33483
Supersedes #33513
cc @apolcyn
Co-authored-by: alto-ruby <altorubys@gmail.com>
The function grpc_rb_call_run_batch has many places that could raise
errors, including in child functions. Since a raised error will longjump
out of the function, it will cause memory leaks since the function
cannot perform any clean up. This commit fixes the issue by wrapping the
whole function in an rb_ensure, which will ensure that a cleanup
function is ran before the error is propagated upwards.
The function grpc_rb_server_request_call has many places that could
raise errors, including in child functions. Since a raised error will
longjump out of the function, it will cause memory leaks since the
function cannot perform any clean up. This commit fixes the issue by
wrapping the whole function in an rb_ensure, which will ensure that a
cleanup function is ran before the error is propagated upwards.
re2 previously failed to compile if:
1. An old `re2` version is installed with a non-standard system prefix,
such as `/opt/local`.
2. The environment variable is set: `CPPFLAGS=-I/opt/local/include`.
Running `make` would result in function prototype mismatches because the
Makefile would previously attempt to use the headers from
`/opt/local/include/re2` before the `third_party/re2/re2` directory.
https://github.com/grpc/grpc/pull/27660 caused `CPPFLAGS` to inherit
from the environment, but this can cause the Makefile to use external
include files for re2 and other libraries if `-I` flags are defined.
This commit reverts to the original behavior of only using
`RbConfig::CONFIG` values to avoid using the wrong headers.
Let's see if this fixes the "Bus error" flakes that have been happening
in CI.
If it does, then we can narrow things down a bit. If flakes continue,
then we can revert this PR.
* src/proto/grpc/core/stats.proto no longer exists
* remove PHP generated file for proto that no longer exists
* run src/ruby/pb/generate_proto_ruby.sh
In Ruby 3.2 the default allocator warns if used with `Data_Wrap_Struct`. This removed the default allocator in those cases.
Ruby discussion: https://bugs.ruby-lang.org/issues/18007
Previously attempting to build the Ruby C extensionwould result in
undefined symbol:
```
Undefined symbols for architecture arm64:
"_ruby_abi_version", referenced from:
-exported_symbol[s_list] command line option
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
As noted in https://github.com/ruby/ruby/pull/6231,
`_ruby_abi_version` is only available for development
versions. `RUBY_PATCHLEVEL` < 0 denotes a development version, while
>= 0 denotes a released version.
Add a check that SEND_STATUS_FROM_SERVER and RECV_MESSAGE are not in the same batch.
This is necessary pre-work for #31204 and implements part of grpc/proposal#336.
Also eliminates fling instead of updating it:
My expectation is nobody has looked at this corner in many years
It's not a benchmark we want: concentrating on a microbenchmark that doesn't include a binding layer caused us to favor designs that emphasized a lightweight core at the expense of a expensive bindings. We should consider the whole.
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Fix Ruby Windows 3.1 (x64-mingw-ucrt) build:
- Pass-in CPPFLAG "GPR_WINDOWS_UCRT" to pickup correct shared object file.
- Rename intermediate legacy SO file from 'grpc_c.64.ruby' to 'grpc_c.64-msvcrt.ruby' for clarity.
- Add artifact target for Ruby x64-mingw-ucrt
* Change CPP flag name
* replace darwin checks in extconf.rb to exclude TruffleRuby
* inherit RANLIB and STRIP from RbConfig, set LDXX
* enable overriding ranlib command in top-level makefile
* ensure the -no_warning_for_no_symbols flag is only used with Apple's ranlib
* don't embed openssl & zlib on truffleruby
* add RbConfig's cppflag to CPPFLAGS when using TruffleRuby
* this ensure the paths to find the system's OpenSSL are set up correctly with TruffleRuby (includes being able to find an OpenSSL installed via Homebrew etc)
* don't statically link standard libraries on Linux with Truffleruby
* This does not work when compiling to bitcode.
* Prefer SIGTERM to SIGQUIT for graceful shutdown in examples
* Overriding SIGQUIT is suboptimal, for example on JVM where it is very
useful to dump the thread stacktraces.
* Keep the rb_tr_abi_version symbol for TruffleRuby in grpc_c.so
* Otherwise TruffleRuby cannot verify the ABI version is correct.
* See https://github.com/oracle/truffleruby/issues/2386
* Use RbConfig::CONFIG['STRIP'] instead of just `strip`
* Use a local variable for apple_toolchain for consistency
* Add a comment about -static-libgcc -static-libstdc++ and TruffleRuby
* Split comment into two for openssl/zlib
Co-authored-by: Nicolas Laurent <nicolas.laurent@oracle.com>
* Having the entire logic of extconf.rb duplicated in 2 files is
extremely inconvenient, also because the Ruby logic in the YAML template
does not get any syntax highlighting or IDE features.
apolcyn