This change adds an experimental ssl_session_cache_lru function to the
Python API that returns an encapsulated grpc_ssl_session_cache (#14483).
Python clients may use this object as an argument for the
grpc.ssl_session_cache channel option if they wish to cache and resume
TLS sessions with a server.
@nathanielmanistaatgoogle and I just had a good session helping me
figure out how to use this API correctly, and this rewording of the
documentation was the result.
All logging in Python so far was done with the root logger, resulting
in logs like: `ERROR:Exception calling application:`. With module-level
loggers, the logs will instead include the module in which the
exception is raised: `ERROR:grpc._server:Exception calling application:`
This removes invocation-side completion queues from the _cygrpc API.
Invocation-side calls are changed to no longer share the same lifetime
as Core calls.
Illegal metadata is now detected on invocation rather than at the start
of a batch (so passing illegal metadata to a response-streaming method
will now raise an exception immediately rather than later on when
attempting to read the first response message).
It is no longer possible to create a call without immediately starting
at least one batch of operations on it. Only tests are affected by this
change; there are no real use cases in which one wants to start a call
but wait a little while before learning that the server has rejected
it.
It is now required that code above cygrpc.Channel spend threads on
next_event whenever events are pending. A cygrpc.Channel.close method
is introduced, but it merely blocks until the cygrpc.Channel's
completion queues are drained; it does not itself drain them.
Noteworthy here is that we drop the cygrpc.Channel.__dealloc__ method.
It is not the same as __del__ (which is not something that can be added
to cygrpc.Channel) and there is no guarantee that __dealloc__ will be
called at all or that it will be called while the cygrpc.Channel
instance's Python attributes are intact (in testing, I saw both in
different environments). This commit does not knowingly break any
garbage-collection-based memory management working (or "happening to
appear to work in some circumstances"), though if it does, the proper
remedy is to call grpc.Channel.close... which is the objective towards
which this commit builds.
The Beta API has an execution date and RPC Framework is but a distant
memory.
This test is flaky with Python 3.5 on Windows! Some mysteries will just
have to remain unsolved...
Currently, `warning: sdist: standard file not found` warning is always
printed when creating source distribution for gRPC python reflection
package. This should stop that.
The thread that watches connectivity on the channel might not
pick up that the server has gone away before the request is
dispatched, and return UNAVAILABLE instead of reconnecting
prior to sending the request. The fundamental solution would
basically be enabling retries in C-core. For now, we opt to
sleep a second to deflake this particular test case.
A gRPC Python client interceptor is passed an instance of a class
that implements the ClientCallDetails interface. The interceptor
can choose to create its own object that implements the interface,
and pass it back to the continuation invoked by the interceptor.
To make it easy to add additional attributes to call details,
without breaking user code that hardcode the attributes required
by the interface, instead of interospecting the object passed
to the interceptor at runtime, and to ease authorship of
interceptors that want to keep some attributes intact and not
care about them, we relax the requirements on the object that
is expected to get passed by the interceptor and let the user
omit some attributes. Omitted attributes will be replaced
by the original value of the attribute given to the interceptor.
grpc.ServicerContext.abort is documented to always raise an exception
to terminate the RPC. The code argument "must not be StatusCode.OK."
However, if you do pass StatusCode.OK, the RPC terminates successfully
on the client side, but returns None.
_server.py: If the user accidentally passes StatusCode.OK, treat it as
StatusCode.UNKNOWN. This is what happens if the user accidentally
passes something that is not a StatusCode instance. Additionally
set details to ''.
_metadata_code_details_test.py: update test to verify the behavior of
abort with invalid codes.
Rather than single classes they are now broken up into class families
with each class containing only those fields and methods that are
needed in the context in which the class is used.
Picks up work from https://github.com/grpc/grpc/pull/10259.
A merge was impossible due to the many sweeping changed that
have occured since I last touched that PR (c++-ization, exec_ctx,
reorganitation of filters, etc).
It is now a family of classes conforming to an interface rather than a
single class no single instance of which makes use of all behavior
scoped to the class.
It also now only uses gRPC Core memory for the time of a single batch
rather than for the entire lifetime of the instance.
When we set the call state to "CANCELLED" after
grpc_cancel_all_calls, we would block other start batch
operations from happening. The rpc_state for the cancelled
call would still be in the server's rpc_states set, but it
would never get removed because there were no active batches
for the call, and the only place we remove from rpc_states is
when a batch completes.
It is better to rely on c-core's cancellation. Once a call
is cancelled, all subsequent ops on that call will return
immediately with a cancellation error.
The RLock() change is due to the possibility that
_on_call_completed
gets invoked immediately when the call has already completed when the
rpc_future callback is created.
gRPC Python required RPCs terminating with non-OK status code to still
return a valid response value after calling set_code, even though the
response value was not supposed to be communicated to the client, and
returning None is considered a programming error.
This commit introduces an alternative mechanism to terminate RPCs by
calling the `abort` method on `ServicerContext` passed to the handler,
which raises an exception and signals to the gRPC runtime to abort the
RPC with the specified status code and details.
A GenericRpcHandler registered on a gRPC Server is not supposed to raise
an exception and if it does so it is considered a programming defect.
However, gRPC is supposed to respond to the client with an UNKNOWN
status code. Previously, this situation was left unhandled and the
client ended up receiving a response with CANCELLED status code.
This commit fixes the issue https://github.com/grpc/grpc/issues/13629.
Rather than allocating gRPC Core memory when instantiated and
retaining it until deleted, gRPC Python's credentials objects now
offer methods to create gRPC Core structures on demand.
"Do not make undocumented and unsupported use of one part of an API
from within the implementation of another part of the same API" is one
of the rules of Abstraction Club. Not sure if it's important enough to
be the first two rules, but it's up there.
Using the presence of the `*_pb2_grpc` module, as opposed to the absence
of the build script (`*_commands` module) is a problematic choice,
because even if a generated file is present, the test infrastructure may
want to regenerate it under a different environment (e.g. different
Python/proto package version). This will ensure the protos always get
recompiled if we have a `*_commands` module present, signaling we are in
a build environment, thereby making the process hermetic.
Previously, a secure server is configured with SSL credentials during
initialization, and those credentials will be used for the lifetime of
the server. If the user wants the server to use new credentials, the
user has to restart the server, resulting in server downtime. This
change enables the user to optionally configure the server with a
"certificiate config fetcher," such that on every new client
connection, the server will call the config fetcher before performing
the handshake, allowing the user application to optionally specify new
certificate configuration for the server to use (the fetcher can
return a "no change" and the server continues to use its current
certificate configuration).
The previous packaging structure exhibited strange behavior of
slowness when trying to use pip to install grpcio-reflection
or grpcio-health-checking in a single line with grpcio-tools.
The root cause seems to be the complicated interaction between
pip and setuptools and the fact that we ship a single .tar.gz
"source" archive for `grpcio_reflection` and
`grpcio_health_checking` packages. `pip` tries to build this
"source" package, and our build process wants to generate
code for the `.proto` files in the package. However, we have
already processed the `.proto` files into `_pb2.py` files in
our artifact build process, and installing `grpcio_tools`
to get `grpcio_{reflection,health_checking}` seems excessive.
The behavior gets worse since `setuptools`, while building
the package from source, tries to fetch `grpcio_tools` from
source and build that too. This takes a while, since it
involves compiling a bunch of native code from `protobuf` and
`grpc` and requires a C compiler to boot.
This commit modifies the Python artifact for the two packages
so that they will not include the raw `.proto` files in the
distribution uploaded to PyPI, nor would they contain the
Python module that does the preprocessing code generation
from the respective .proto files. Instead, a specific code
path is taken when the generated `_pb2_grpc` Python module is
not present in the package to provide such functionality
when built from the gRPC git repository (and hence when built
from our CI infrastructure.)
This commit is a rebase of an old contribution with minor formatting
edits (cf. https://github.com/grpc/grpc/pull/8663)
[Original Author]
Masood Malekghassemi <atash@google.com>
Date: Mon Nov 7 14:49:09 2016 -0800
Contributor is a Xoogler and the contribution is owned by Google Inc.
as per the copyright assignment agreement with the original author,
as it was drafted during their employment with Google Inc.
Beta code elements are not generated at all in _pb2_grpc.py files.
This duplicates a lot of the in-test code generation done in
_split_definitions_test. In a future clean-up we may want to
deduplicate the common behavior, put it in a module available to all
other tests, and do all of our testing of generated code with in-test
code generation.
I made this mistake in 2010985ab2 but
only with yesterday's release of six 1.11.0 has it started failing
("TypeError: metaclass conflict: the metaclassof a derived class must
be a (non-strict) subclass of the metaclasses of all its bases").
We were mistaken before when we were testing _pb2.py files being
generated in one directory and _pb2_grpc.py files being generated in
another directory. Sure, that was a thing our code generator could do,
but because of the way paths and packages work in Python it wasn't a
realistic use case for actual users.
This test now tests _pb2.py files and _pb2_grpc.py files being
generated either together or independently of one another, and if
independently, in either order. Looking forward to an eventual
py_grpc_library Bazel rule, that's what actually matters.
- Move it out of the "unit" package, as it's not itself a unit test.
- Suffix the test class with "Test" as we do with every other subclass
of unittest.TestCase.
- Add a larger-than-we'll-need-any-time-soon maxDiff so that failures
are fully described.
- Relax the assertion from assertListEqual to assertSequenceEqual
since we don't actually care whether or not the sequences being
compared are list instances.
- Change the order of the assertions arguments to match the
"<expected>, <actual>" convention used in our assert*Equal calls
elsewhere throughout the test corpus.
- Internal implementation simplifications.
Were this not done these would break when the default behavior of gRPC
Python Protoc Plug-In is changed to be the put-gRPC-code-elements-only-
in-_pb2_grpc.py-files behavior that currently happens only when the
grpc_2_0 flag is passed.
Mehrdad Afshari