Commit Graph

300 Commits

Author SHA1 Message Date
David Klempner fd99dd3475 Cap the size of the poll kick freelist.
This caps the size of the poll kick freelist at the arbitrary value of
25.
2015-01-20 17:18:39 -08:00
Craig Tiller bcfa267b5f Release server creds 2015-01-19 18:23:30 -08:00
Craig Tiller f97170716f Merge github.com:google/grpc into sec 2015-01-19 16:19:19 -08:00
Craig Tiller 64761f8af6 clang-format 2015-01-19 15:33:02 -08:00
Craig Tiller 64f65458e3 Allow secure connections with fling server 2015-01-19 15:18:10 -08:00
Craig Tiller ecd49345b5 Use clang-format-3.5 2015-01-18 14:36:47 -08:00
Craig Tiller 59140fc116 Explicitly name the target language.
Instead of C++ or not C++, name C++ and C, in preparation for being able to name other languages in the future.
2015-01-18 10:12:17 -08:00
vjpai a537ae465b Merge pull request #70 from ctiller/benchmark
Opportunistically use perftools if installed.
2015-01-16 16:44:49 -08:00
David Klempner 7f3ed1eda1 Factor out the pollset kicking mechanism and eliminate sharding
This change pulls out a separate pollset_kick module, which currently
uses a freelist of pipes dynamically assigned to pollsets when they
enter polling rather than the previous racy sharding mechanism.

We ultimately may wish to eliminate the dynamic assignment for multipoll
sets, but this should be sufficient for the moment.
2015-01-16 15:42:31 -08:00
Craig Tiller 29fa1c2fa4 Compile fix 2015-01-16 15:26:11 -08:00
Craig Tiller 5188535a26 Merge branch 'new_invoke_api' of github.com:google/grpc into update-api 2015-01-16 15:24:17 -08:00
Craig Tiller 33a0b83af3 clang compile fixes 2015-01-16 15:14:10 -08:00
Yang Gao aa6f993841 Merge pull request #88 from ctiller/shutdown
Add an optional server shutdown event.
2015-01-16 14:05:27 -08:00
Craig Tiller 4ffdcd56ce Add an optional server shutdown event.
Allows servers to expedite shutdown once all in-progress calls are
completed. We may want to eventually remove grpc_server_shutdown in
preference to this.
2015-01-16 11:34:55 -08:00
Craig Tiller 4853791a84 Build fix 2015-01-16 09:57:49 -08:00
Craig Tiller 7b01878736 Fix returned status
The client should return status deadline exceeded when the deadline is
exceeded (not cancelled status)
2015-01-16 09:53:39 -08:00
Craig Tiller 7132d51b2e Include some useful help text 2015-01-16 09:00:12 -08:00
Craig Tiller c8c8bfa403 Merge branch 'new_invoke_api' of github.com:google/grpc into update
Conflicts:
	src/core/surface/call.c
	test/core/end2end/tests/thread_stress.c
	tools/run_tests/run_tests.py
2015-01-15 22:57:02 -08:00
Julien Boeuf b222b4d978 Changing size type in Transport Security (TSI) from uint32_t to size_t:
Will remove grpc impedance mismatch.
2015-01-15 22:47:49 -08:00
Craig Tiller 297fafa078 Opportunistically use perftools if installed.
Allows us to collect profiles of the open source build using gprof.
2015-01-15 15:46:39 -08:00
Craig Tiller 3eef2c4bcb clang-format 2015-01-15 11:37:54 -08:00
Craig Tiller 32946d37e0 () --> (void) 2015-01-15 11:37:30 -08:00
Craig Tiller d7f2c2b1c7 clang-format 2015-01-15 10:18:47 -08:00
Craig Tiller a2075b90d7 Remove calls to grpc_call_accept from C core tests.
This call is deprecated.
2015-01-15 10:16:35 -08:00
Craig Tiller 408c7383cc clang-format 2015-01-14 16:25:47 -08:00
Craig Tiller ca75796811 Fling test uses grpc_call_invoke 2015-01-14 16:17:05 -08:00
Craig Tiller 240d02c076 Echo test uses grpc_call_invoke 2015-01-14 16:16:52 -08:00
Craig Tiller 09b637538f clang-format 2015-01-14 13:02:09 -08:00
Craig Tiller 465e2e11d1 Merge github.com:google/grpc into api 2015-01-14 13:01:46 -08:00
Yang Gao 5680ff4687 Run clang-format-all.sh 2015-01-14 12:14:21 -08:00
Craig Tiller e1e2475b69 Unify signal handling in C 2015-01-14 11:21:00 -08:00
Craig Tiller 6d43445a1a Merge github.com:google/grpc into api 2015-01-14 11:19:09 -08:00
Nicolas "Pixel" Noble 80d68c09c4 Fixing test_config.c
Now all the gpr tests build and run under win32.
2015-01-13 20:14:43 -08:00
Craig Tiller 5c35008ea1 Unify signal handling in C 2015-01-13 17:26:27 -08:00
Craig Tiller bf444937c6 Fix ordering problem leading to flaky test 2015-01-13 17:13:08 -08:00
Craig Tiller fc84e7b28e Fix tag matching bug 2015-01-13 16:49:44 -08:00
Craig Tiller 9c0307bd4f Update test to grpc_call_invoke 2015-01-13 16:49:24 -08:00
Craig Tiller 97f7fca5ff Remove calls to grpc_call_start_invoke 2015-01-13 16:36:31 -08:00
Craig Tiller cbbac9b10d Compile fix 2015-01-13 16:13:43 -08:00
Craig Tiller 40fc7a6632 clang-format codebase 2015-01-13 16:11:58 -08:00
Craig Tiller 375605b65f Merge github.com:google/grpc into api 2015-01-13 16:11:30 -08:00
Craig Tiller 80fa15c151 Moving prototype from Google to GitHub
I'd started some prototyping work on this change before the move to
GitHub; this change restores things.
2015-01-13 16:10:49 -08:00
Craig Tiller 21b19a552c Fix formatting 2015-01-13 12:39:32 -08:00
Craig Tiller 1ee7cb0c0b Merge github.com:google/grpc into format 2015-01-13 12:39:00 -08:00
Craig Tiller b5dcec5a2e clang-format codebase 2015-01-13 11:13:42 -08:00
Craig Tiller 770f60a475 Merge github.com:google/grpc into testcpp 2015-01-12 17:44:43 -08:00
nnoble 5f2ecb355d Splitting grpc_test_util into gpr_test_util.
Change on 2015/01/12 by nnoble <nnoble@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83801564
2015-01-12 17:36:57 -08:00
ctiller 570d1f4425 Return bound port number from grpc_server_add_http2_port.
Allows tests to bind to port 0 and still have clients connect to them.
	Change on 2015/01/12 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83800669
2015-01-12 17:36:57 -08:00
Craig Tiller edd10d2596 Make this file usable from C++ 2015-01-12 16:40:51 -08:00
hongyu 99e317a8f8 grpc census hash_table test coverage improvements.
Change on 2015/01/12 by hongyu <hongyu@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83786701
2015-01-12 14:11:52 -08:00
ctiller 00297df63b Ensure flow control callbacks happen outside the transport lock.
Split encoding into two phases: a collection phase to decide on what is allowed
(by flow control) to be sent, and a framing phase when the data is actually
sent. Perform the second phase outside of the transport mutex (but serially,
guarded by t->writing) and make flow control callbacks during that phase.

This will allow us to make further transport level calls in response to flow
control callbacks, and will be needed by the forthcoming async api for C++.
	Change on 2015/01/12 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83774409
2015-01-12 14:03:28 -08:00
ctiller 0cd69562fd Allow logging mechanism to be overridden by applications.
This necessitated the removal of gpr_vlog to keep a clean interface.
	Change on 2015/01/09 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83634996
2015-01-12 11:22:02 -08:00
hongyu 440d3d8198 Grpc census: v0 trace_printf() implementation in C.
Change on 2015/01/09 by hongyu <hongyu@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83626942
2015-01-12 11:22:01 -08:00
Nicolas Noble 362b9dcfc9 Merge github.com:google/grpc 2015-01-09 18:05:07 -08:00
ctiller 8bf337bedf Fix test under msan
Change on 2015/01/09 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83609353
2015-01-09 17:55:21 -08:00
hongyu 24200d3cbc C implementation of Census trace store and stats store for grpc C lib.
Change on 2015/01/08 by hongyu <hongyu@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83556470
2015-01-09 17:52:43 -08:00
ctiller 58393c20bc Remove libevent.
Fixed any exposed bugs across the stack.

Add a poll() based implementation. Heavily leverages pollset infrastructure to allow small polls to be the norm. Exposes a mechanism to plug in epoll/kqueue for platforms where we have them.

Simplify iomgr callbacks to return one bit of success or failure (instead of the multi valued result that was mostly unused previously). This will ease the burden on new implementations, and the previous system provided no real value anyway.

Removed timeouts on endpoint read/write routines. This simplifies porting burden by providing a more orthogonal interface, and the functionality can always be replicated when desired by using an alarm combined with endpoint_shutdown. I'm fairly certain we ended up with this interface because it was convenient to do from libevent.

Things that need attention still:
- adding an fd to a pollset is O(n^2) - but this is probably ok given that we'll not
use this for multipolling once platform specific implementations are added.
- we rely on the backup poller too often - especially for SSL handshakes and for client
connection establishment we should have a better mechanism ([] []
- Linux needs to use epoll for multiple fds, FreeBSD variants (including
Darwin) need to use kqueue. ([] []
- Linux needs to use eventfd for poll kicking. ([]
	Change on 2015/01/07 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83461069
2015-01-09 17:27:32 -08:00
ctiller e4b409364e Add a --forever flag, to continuously run tests as things change.
Change on 2015/01/07 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83451760
2015-01-09 17:23:18 -08:00
Craig Tiller a6899be634 Fix a compiler warning. 2015-01-09 12:54:01 -08:00
ctiller 1a277ecd93 Remove libevent.
Fixed any exposed bugs across the stack.

Add a poll() based implementation. Heavily leverages pollset infrastructure to allow small polls to be the norm. Exposes a mechanism to plug in epoll/kqueue for platforms where we have them.

Simplify iomgr callbacks to return one bit of success or failure (instead of the multi valued result that was mostly unused previously). This will ease the burden on new implementations, and the previous system provided no real value anyway.

Removed timeouts on endpoint read/write routines. This simplifies porting burden by providing a more orthogonal interface, and the functionality can always be replicated when desired by using an alarm combined with endpoint_shutdown. I'm fairly certain we ended up with this interface because it was convenient to do from libevent.

Things that need attention still:
- adding an fd to a pollset is O(n^2) - but this is probably ok given that we'll not
use this for multipolling once platform specific implementations are added.
- we rely on the backup poller too often - especially for SSL handshakes and for client
connection establishment we should have a better mechanism ([] []
- Linux needs to use epoll for multiple fds, FreeBSD variants (including
Darwin) need to use kqueue. ([] []
- Linux needs to use eventfd for poll kicking. ([]
	Change on 2015/01/07 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83461069
2015-01-08 13:01:56 -08:00
nnoble 9f312b49e8 Fixing include paths for cmdline. 2015-01-07 01:29:22 -08:00
nnoble c87b1c533f Adding plugins to the mako rendering system.
Change on 2015/01/05 by nnoble <nnoble@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83302581
2015-01-06 17:40:30 -08:00
ctiller b3e3317ae8 Seed random numbers in each process based on pid.
This causes get_unused_port_or_die to try different port numbers in each
concurrently running tests, and consequently enables some parallelism of test
processes.
	Change on 2015/01/05 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83289374
2015-01-05 17:43:10 -08:00
klempner cd45aadc10 Fix the socket pair fixtures to include the http filters.
Without this these tests start failing if the server starts checking that it
received the necessary headers.
	Change on 2014/12/23 by klempner <klempner@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82748741
2014-12-29 17:29:30 -08:00
chenw 97fd9e56d2 Add SSL root for production GFE.
Change on 2014/12/19 by chenw <chenw@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82554526
2014-12-29 17:06:52 -08:00
ctiller 3bf466fb6c Port [] alarm management to GRPC.
This change implements a platform independent alarm manager in alarm.c.
It's integrated with iomgr, and some tests are cleaned up.

The alarm implementation itself is a fairly direct port of LazyAlarmList from eventmanager.
SpinLock has been replaced for now with gpr_mu, and other atomic operations have been dropped (again, for now).
A majority of tests have been ported.
	Change on 2014/12/19 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82551363
2014-12-29 17:05:35 -08:00
jboeuf 1a809c0ebb Adding support for service account credentials.
- Tested end to end with a JSON key I generated for my account using the
fetch_oauth2 binary.
- The same fetch_oauth2 binary can get a token from the GCE metadata service on a VM in cloud.
	Change on 2014/12/19 by jboeuf <jboeuf@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82548689
2014-12-29 17:03:04 -08:00
ctiller 336bf21127 Fix opensource build test.
It's a string of unit tests, so size should actually be small. Setting timeout
to long means we have 15 minutes to test everything, which should fit for now.

Also fixes a compile error.

Once this goes in, TAP should start running our tests again.
	Change on 2014/12/17 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82395309
2014-12-19 13:13:37 -08:00
pmarks b74550655a gRPC: Avoid AF_INET6 sockets when the ::1 loopback address doesn't exist.
On Linux with disable_ipv6=1, we can create sockets bound to [::]:port, yet
connecting to that address triggers an Unreachable error.  Since IPv6 is
useless on such machines, it's cleaner to turn it off than expose users to a
half-broken state.
	Change on 2014/12/17 by pmarks <pmarks@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82387437
2014-12-19 13:11:09 -08:00
ctiller cd0569a16c Fix include guards.
Change on 2014/12/17 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82354016
2014-12-17 15:53:55 -08:00
ctiller 2bbb6c4311 Remove endpoint/ directories.
Fold endpoint interface into iomgr, move secure_endpoint into security/.
This will make it easier for endpoint to rely on some iomgr defined types (like
pollset).
	Change on 2014/12/17 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82338036
2014-12-17 15:49:42 -08:00
jboeuf ab4f91458b Adding JWT generation and signing code for service accounts credentials.
Change on 2014/12/16 by jboeuf <jboeuf@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82279249
2014-12-16 17:26:10 -08:00
ctiller 769b70ba9d Make alarm initialization take a timespec now.
This will allow combining of gpr_now() calls, and make it easier to test the
alarm code in the future.
	Change on 2014/12/16 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82246768
2014-12-16 17:13:37 -08:00
ctiller c99774fc7b Remove a (very) deprecated file.
Change on 2014/12/15 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82201880
2014-12-16 17:08:15 -08:00
ctiller 2845cade4c Trailing metadata support.
Actually exposing it from the C API.
Also removing grpc_status, since it's mostly useless.
	Change on 2014/12/15 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82180546
2014-12-16 16:57:49 -08:00
ctiller c6d61c4dd6 GOAWAY & Reconnection support.
Clients stay connected to a server after it shutdowns until all active calls
have completed, and then they drop.

After a GOAWAY or a disconnect, clients will attempt to re-resolve and
reconnect to the server.
	Change on 2014/12/15 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82178515
2014-12-15 15:39:03 -08:00
ctiller c1ddffbf1f Add time averaged stats for future alarm list work.
This is a trivial C89-ification of the []2 implementation of the same idea.
Indeed the implementation files and tests have been branched from the [] versions.
	Change on 2014/12/15 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82168822
2014-12-15 14:47:44 -08:00
ctiller c5abe59fe1 Simplify alarm API.
This change removes a corner case that needs to be handled (what happens if you
add an alarm twice?). This corner case was never used, nor tested.
	Change on 2014/12/15 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82168179
2014-12-15 14:46:23 -08:00
aveitch 482a5be0b9 Fix a bug in base-log, and add test.
The tests compute usable space in the log, but do so using subtraction on unsigned values and did not correctly account for number of blocks used per-core. This could lead underflow, and an incorrect space calculation.

In addition, testing showed that with current defaults, tests would fail on a system with > ~64 cores. This can be alleviated through use of a larger log. An explicit
check in the log initialization has been added for this case, and the log size increased if necessary.
	Change on 2014/12/15 by aveitch <aveitch@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82154432
2014-12-15 14:36:22 -08:00
ctiller 5e04b1376d Add a test for useful.h.
Additionally add rotl, rotr which will be needed to implement some hashing
functions shortly.
	Change on 2014/12/15 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82149559
2014-12-15 14:32:14 -08:00
klempner ee3ebe1bb9 Change the low level ping pong benchmark to run all tests by default.
Change on 2014/12/12 by klempner <klempner@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82026092
2014-12-15 14:30:53 -08:00
ctiller 33023c4bac Binary header encoding.
Also fixes a rather embarrassing bug in bin_encoder.c.
	Change on 2014/12/12 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82024795
2014-12-15 14:28:28 -08:00
jboeuf befd26501a Adding base utils for JWT service account workflow. OpenSSL base64 decoding is
a disaster and does not support url_safe which we need for the JWT.
	Change on 2014/12/12 by jboeuf <jboeuf@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82020601
2014-12-12 16:36:06 -08:00
nnoble 8a67b5c281 Tweaking log calls a bit.
-) Introducing gpr_vlog so to spare a few vsprintf later (at least one for now)
-) Renaming statistics/log.* to statistics/census_log.* to avoid collisions.
	Change on 2014/12/12 by nnoble <nnoble@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81995756
2014-12-12 16:20:22 -08:00
jtattermusch 94204dda83 Fix ipv6.c build breakage on Windows.
Change on 2014/12/11 by jtattermusch <jtattermusch@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81910401
2014-12-11 15:11:44 -08:00
jtattermusch 57c6f0cade Add batch file that builds and executes all GPR tests.
Removed bad format test to fix gpr_string_test.
Also took care of some of the MS compiler warnings when compiling gpr_test_util.
	Change on 2014/12/11 by jtattermusch <jtattermusch@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81900639
2014-12-11 15:11:44 -08:00
ctiller 430c4996e8 Cache encoded binary data on the metadata string object.
Change on 2014/12/11 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81882793
2014-12-11 15:11:44 -08:00
pmarks 465554e5b6 Add a grpc_ipv6_loopback_available() test utility, and only run IPv6-specific
tests on machines that support AF_INET6 sockets bound to ::1.

Listening on :: or connecting to a mapped address should always work.
	Change on 2014/12/10 by pmarks <pmarks@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81837056
2014-12-11 15:11:44 -08:00
klempner c9e80fb26a Patch shutdown during write test to not fail or do a racy destroy if the write
completes successfully.
	Change on 2014/12/10 by klempner <klempner@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81831655
2014-12-11 15:11:44 -08:00
ctiller f962f5264b Tell call/channel op handlers who is invoking them.
This change adds a parameter to all op handlers specifying the invoking filter.
It will be used to allow client_channel to distinguish which child channel is
disconnecting or going away.
	Change on 2014/12/10 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81823231
2014-12-11 15:11:44 -08:00
ctiller 48b5a4586a Advertise h2-16, h2-15, h2-14, and accept any of them.
(Fixing the rollback from earlier - we were passing '1' as the protocol count, not num_alpn_protocols)
	Change on 2014/12/10 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81783755
2014-12-10 13:22:22 -08:00
ctiller 1ef8175bc5 Automated g4 rollback of changelist 81719124.
*** Reason for rollback ***

Breaks interop tests

*** Original change description ***

Advertise h2-16, h2-15, h2-14, and accept any of them.

***
	Change on 2014/12/09 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81723283
2014-12-09 17:47:34 -08:00
ctiller 84518b4b85 Advertise h2-16, h2-15, h2-14, and accept any of them.
Change on 2014/12/09 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81719124
2014-12-09 16:20:55 -08:00
ctiller 18b49ab914 Introducing iomgr.
Move eventmanager and platform dependent endpoint functionality into a single
library called 'iomgr'.

This is primarily to prepare for a Windows port - where posix socket semantics
lead to poor quality code.

Mostly this is a code movement CL, with some small changes to help prepare the
way for porting:

- em style fd objects can only be held internally in iomgr, and own their memory
- added grpc_iomgr_create_endpoint_pair() to accomodate the common pattern of
creating a tcp endpoint from the output of socketpair - this will help keep
our tests portable
- separated em alarm interface into a separate file, as this part of event
manager is needed higher up the stack
- made the eventmanager bits a true singleton, simplifying API's across the
stack as there's no longer a reason to carry a pointer there.

Initial design document is here:
https://docs.google.com/document/d/1VmafcHvvrP5kwtQkz84R5yXF7u7fW-9Pn0bkSUQHDt8/edit?disco=AAAAARNByxg
	Change on 2014/12/09 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81716456
2014-12-09 16:20:55 -08:00
nathaniel 5287817527 Strip "_test" from the name of thread_stress_test.c.
Having a BUILD target named chttp2_fullstack_thread_stress_test_test
is deceptive as it is a direct test of the system rather than a test
of a test of the system.
	Change on 2014/12/09 by nathaniel <nathaniel@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81688649
2014-12-09 16:20:55 -08:00
jtattermusch 97fb3f6481 Add mako templates to generate MS Visual Studio projects.
Current state for Windows platform:
-- gpr and grpc_test_util can be compiled
-- gpr_log_test and gpr_cmdline_test pass
	Change on 2014/12/08 by jtattermusch <jtattermusch@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81615574
2014-12-08 18:50:13 -08:00
klempner 5ef51949bc Make em destroy underlying descriptor objects (and their corresponding libevent
objects and closing the actual fd) asynchronously in the poller thread where
they won't race with libevent internals while polling.

This requires splitting the actual descriptor data from the application owned
handle, because the former needs to have a longer lifetime.

This CL also hacks dualstack_socket_test which is legitimately sensitive to the
delayed close of the listening socket.
	Change on 2014/12/08 by klempner <klempner@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81603300
2014-12-08 18:50:12 -08:00
nnoble 0c475f0ad7 Updating moe_db.txt with the latest equivalence since the ruby import changed the exported structure.
Change on 2014/12/01 by nnoble <nnoble@google.com>
-------------
new [] file for grpc testing.
	Change on 2014/12/02 by donnadionne <donnadionne@google.com>
-------------
Fix unfinished calls in thread_stress_test.

Previously we had an early return if we cancelled a stream part way through a
message. Correct this, so that half close and full close signals are propagated
up the stack correctly so that higher level state machines can see the
termination.
	Change on 2014/12/02 by ctiller <ctiller@google.com>
-------------
Remove dependency on internal C code.
	Change on 2014/12/02 by ctiller <ctiller@google.com>
-------------
Turn off the flaky bit from thread_stress_test.
	Change on 2014/12/02 by ctiller <ctiller@google.com>
-------------
Add test cases of empty request/response, request streaming, response streaming, and half duplex streaming.

Bring up the GFE/ESF for mannual test:
[] build java/com/google/net/[]/testing/integration/hexa:server_components_env
[]-bin/java/com/google/net/[]/testing/integration/hexa/server_components_env --manual --rpc_port=25000 --use_autobahn
	Change on 2014/12/02 by chenw <chenw@google.com>
-------------
Make echo/server.c and fling/server.c shutdown cleanly on SIGINT, and update
the relevant tests to exercise this mechanism.

Now "[] coverage" and the memory leak detector are able to see into the
server processes.
	Change on 2014/12/02 by pmarks <pmarks@google.com>
-------------
Allow the # of channels to be configurable in this performance test. The threads will use the channels in statically-defined round-robin order (not based on when RPCs complete on any channel). The interesting cases are #channels=1 or #channels=#threads (we previously only had the latter case)
	Change on 2014/12/02 by vpai <vpai@google.com>
-------------
Fixed a typo and reworded a comment.
	Change on 2014/12/02 by gnezdo <gnezdo@google.com>
-------------
Require the grpc_call in this ClientContext to be NULL before allowing set_call to be invoked. Otherwise, it's an indication of a leak somewhere.
	Change on 2014/12/02 by vpai <vpai@google.com>
-------------
Correctly return status other than ok and add a test for it.
	Change on 2014/12/02 by yangg <yangg@google.com>
-------------
Better C++ guards for grpc_security.h
	Change on 2014/12/02 by nnoble <nnoble@google.com>
-------------
Use nullptr instead of NULL for consistency.
	Change on 2014/12/02 by vpai <vpai@google.com>
-------------
Updates the ruby gRPC service class to require the serialization method to be
a static method

- this brings it inline with the proto3 ruby API
- it adds a monkey patch to allow existing proto (beefcake) to continue working.
	Change on 2014/12/02 by temiola <temiola@google.com>
-------------
Adding a buildable unit to the blue print file.

Added the buildable unit as its name will be usesd as tap project id.

This test will fail right away in tap until tests are actually added.
	Change on 2014/12/02 by donnadionne <donnadionne@google.com>
-------------
Move interop ESF C++ server from Java to grpc directory.

Tests passed:
[] test javatests/com/google/net/[]/testing/integration/hexa/...
[] test net/grpc/testing/interop/esf_server/...
	Change on 2014/12/02 by chenw <chenw@google.com>
-------------
Return a lame channel as opposed to NULL when secure channel creation fails.

- Looks like we're going to need something similar server-side.
- I changed the prototype of the lame client channel factory to take an
explicit void as I think this is better practice in C. Let me know if you
disagree and I will revert these changes.
	Change on 2014/12/02 by jboeuf <jboeuf@google.com>
-------------
Putting ALPN support where it belongs.
	Change on 2014/12/02 by jboeuf <jboeuf@google.com>
-------------
GOAWAY send path.

Sends a GOAWAY frame when shutting down.
This is not read and understood yet.
	Change on 2014/12/03 by ctiller <ctiller@google.com>
-------------
Adds support for secure channels and servers.

- wraps new C apis (credentials, server_credentials) and Server#add_secure_http_port
- adds tests to ensure credentials and server credentials can be created
- updates client_server_spec to run the client_server wrapper layer end-to-end tests using a secure channel
	Change on 2014/12/03 by temiola <temiola@google.com>
-------------
Fix existing issues regarding out of order events.

At the client side, using pluck as the client_metadata_read can happen anytime after invoke.

At the server side, allow halfclose_ok and rpc_end to come in reverse order.
	Change on 2014/12/03 by yangg <yangg@google.com>
-------------
Don't track coverage of tests.
	Change on 2014/12/03 by ctiller <ctiller@google.com>
-------------
Change UnaryCall to conform standard test requirement of LargeUnaryCall.
	Change on 2014/12/03 by yangg <yangg@google.com>
-------------
updating alpn version to h2-15 ensure all interop are on the same version and working.

Java and go are not ready for h2-16 yet.
	Change on 2014/12/03 by donnadionne <donnadionne@google.com>
-------------
Add config to bring echo server in [].

This is used to test production GFE as its bckend.
	Change on 2014/12/03 by chenw <chenw@google.com>
-------------
In preparation for fixing shutdown race issues, change em to take ownership of
the file descriptor. Add an API to grpc_tcp to take an already created
grpc_em_fd object, and change tcp_client to use that API.

This is needed because otherwise an em user's close() of the file descriptor
may race with libevent internals. That's not an issue yet because destroy()
frees the events inline, but that can't be done safely if there is a concurrent
poller.
	Change on 2014/12/03 by klempner <klempner@google.com>
-------------
Fixing TAP opensource build

We don't want to compile and run C++ tests in the C target.
	Change on 2014/12/03 by nnoble <nnoble@google.com>
-------------
Move and separate interop tests by languages.

Small fixes to the test runner.

Improving logging.
	Change on 2014/12/03 by donnadionne <donnadionne@google.com>
-------------
Fixing the opensource build:

-) The C/C++ split wasn't done up to the 'dep' target level
-) The alpn.c file was missing from build.json
	Change on 2014/12/03 by nnoble <nnoble@google.com>
-------------
Adding blue print files after projects exist
	Change on 2014/12/03 by donnadionne <donnadionne@google.com>
-------------
Refactor StreamContext using the new completion_queue_pluck API.

The dedicated the poller thread has been removed.
This CL keeps the current behavior to make it short. There is one following to
make it usable for both client and server.

The tags for pluck is based on the address of this StreamContext object for potential debug use.

The Read/Write and Wait cannot be called concurrently now and this might need to be fixed.
	Change on 2014/12/03 by yangg <yangg@google.com>
-------------
Binary encoding utilities.

Support base64 encoding, HPACK static huffman encoding, and doing both at once.
	Change on 2014/12/03 by ctiller <ctiller@google.com>
-------------
Enforce Makefile regeneration in presubmits.
	Change on 2014/12/03 by ctiller <ctiller@google.com>
-------------
Make CloseSend() send a real zero-length control message to indicate EOS.
	Change on 2014/12/03 by zhaoq <zhaoq@google.com>
-------------
Prefer to create dualstack sockets for TCP clients and servers, with automatic
fallback for environments where IPV6_V6ONLY can't be turned off.
	Change on 2014/12/03 by pmarks <pmarks@google.com>
-------------
Add opensource path to build targets.

Ensure that MOE is going to run.
	Change on 2014/12/03 by ctiller <ctiller@google.com>
-------------
Add PingPong test case. Delete FullDuplex test case. The latter is not specified for client in

https://docs.google.com/document/d/1dwrPpIu5EqiKVsquZfoOqTj7vP8fa1i49gornJo50Qw/edit#
	Change on 2014/12/03 by chenw <chenw@google.com>
-------------
Make generate_projects.sh check out the generated targets.
	Change on 2014/12/03 by ctiller <ctiller@google.com>
-------------
rspec cleanup

- stops declaring specs within the GRPC module
- splits Bidi streaming specs into a separate test suite

adding tests in the GRPC module was a mistake, it pollutes the module and can
affect other tests that run later by the test runner

the bidi tests are currently flaky, having them run in their own test suite
allows having two separate continuous builds (once ruby gRPC is on GitHub),
one that includes bidi where we tolerate flakiness, and another that does not,
where there should be no flakiness at all
	Change on 2014/12/03 by temiola <temiola@google.com>
-------------
Adding support for composite and IAM credentials.

- For now, we don't do any checks on credentials compatibility in the
composite credentials. Maybe we'll add that later.
- Refactored the end to end security tests so that we always use the public API
(except for the fake security context which is not exposed).
	Change on 2014/12/03 by jboeuf <jboeuf@google.com>
-------------
Make GPR library buildable in Visual Studio 2013.
	Change on 2014/12/04 by jtattermusch <jtattermusch@google.com>
-------------
Adds codegen for ruby

This is being added now that ruby's proto and grpc apis are defined and stable
	Change on 2014/12/04 by temiola <temiola@google.com>
-------------
Prevent NewStream() from sending negative or 0 timeout.
	Change on 2014/12/04 by zhaoq <zhaoq@google.com>
-------------
Add a grpc_sockaddr_to_string() function, and use it when logging bind
failures.  Also improve const-correctness in some earlier code.

I'm not certain whether inet_ntop() will need any platform-specific
implementations, but for now the compiler offers no complaints.

Demo:
$ []-bin/net/grpc/c/echo_server 1.2.3.4:80
... tcp_server.c:139] bind addr=[::ffff:1.2.3.4]:80: Permission denied
	Change on 2014/12/04 by pmarks <pmarks@google.com>
-------------
Refactoring - moves c wrapped classes to a submodule Google::RPC::Core

- this allows for an explicit rule when reading through gRPC ruby code for telling
when an object is pure ruby or wrapped C
	Change on 2014/12/04 by temiola <temiola@google.com>
-------------
Fixes the bidi_call

[]
	Change on 2014/12/04 by temiola <temiola@google.com>
-------------
Fixing dev build when activating surface traces.
	Change on 2014/12/04 by nnoble <nnoble@google.com>
-------------
Updates the tests to reflect that fact that some Credentials compose works.
	Change on 2014/12/04 by temiola <temiola@google.com>
-------------
Making the generate_project_test actually do something.
	Change on 2014/12/04 by nnoble <nnoble@google.com>
-------------
Rename "esf_server" to "[]4_server". Delete "test_sever" from Java directory.
	Change on 2014/12/04 by chenw <chenw@google.com>
-------------
Added PHP client interop tests. Tested large_unary against the C++ server.
	Change on 2014/12/04 by mlumish <mlumish@google.com>
-------------
Refactor grpc_create_dualstack_socket() by pulling the setsockopt into its own
function.  This separates the magic test flag from the real fallback logic.
	Change on 2014/12/04 by pmarks <pmarks@google.com>
-------------
Fixes the type of the constant used for test cert hostname
	Change on 2014/12/04 by temiola <temiola@google.com>
-------------
Disabling these tests as they're causing flakiness.
	Change on 2014/12/04 by ctiller <ctiller@google.com>
-------------
Change intptr --> uintptr.

Handles the case where a void* turns into a negative number, which then gets
hashed into a negative bucket and segfaults.
	Change on 2014/12/04 by ctiller <ctiller@google.com>
-------------
Add a test fixture to force parsers to handle one byte at a time.

This should expand coverage and hopefully prevent errors at some point (it
seems to pass out of the box for now though).
	Change on 2014/12/04 by ctiller <ctiller@google.com>
-------------
The code generator isn't +x.
	Change on 2014/12/04 by ctiller <ctiller@google.com>
-------------
Updates math_client and math_server to allow construction using crednetials

By:
- Extending rpc_server constructor so that it takes a credentials keyword param
- Extending client_stub constructor so that it takes a credentials keyword param
	Change on 2014/12/04 by temiola <temiola@google.com>
-------------
Format output a little more nicely.

Print each line of output separately - previously logging.info was truncating this at some maximum length, and logs were getting lost.
	Change on 2014/12/04 by ctiller <ctiller@google.com>
-------------
Up timeout for this test.

Under TSAN, if we process one byte at a time, this timeout can be reached - and I think this is the cause of the following flake:
[]
	Change on 2014/12/05 by ctiller <ctiller@google.com>
-------------
Adding more error logging for ssl.
	Change on 2014/12/05 by jboeuf <jboeuf@google.com>
-------------
Read path for goaway.

Still need to add hooks to deprecate a channel on the client side when goaway
is received.
	Change on 2014/12/05 by ctiller <ctiller@google.com>
-------------
Separate accept() into server_accept() and server_end_of_initial_metadata().

This allows servers to initiate reads before finishing writing metadata.
	Change on 2014/12/05 by ctiller <ctiller@google.com>
-------------
Fix for breakage 11512317 - adding missing test files.
	Change on 2014/12/05 by nnoble <nnoble@google.com>
-------------
grpc c++ server side streaming support.

This is based on [] There is a lot of room to clean up the internal implementation which may require refactoring of CompletionQueue. The current cl serves as a working implementation with the missing interfaces.

The sample generated files are included and will be removed before submitting.
	Change on 2014/12/05 by yangg <yangg@google.com>
-------------
Changed to the latest timeout format again (search "grpc-timeout" in [] for the spec).
	Change on 2014/12/05 by zhaoq <zhaoq@google.com>
-------------
Fixing opensource build.
	Change on 2014/12/05 by nnoble <nnoble@google.com>
-------------
Making absolutely sure we can do the moe export by adding a sh_test for it.
	Change on 2014/12/05 by nnoble <nnoble@google.com>
-------------
Change :scheme psuedo-header from "grpc" to "http" or "https".
	Change on 2014/12/05 by zhaoq <zhaoq@google.com>
-------------
Add server credential wrapping for c++ server. It only wraps ssl and []2 for now.

The ServerCredentials class and the factory class are in a similar fashion as
client side wrapping. The difference is the factory method returns shared_ptr
instead of unique_ptr as the server builder needs to keep a reference to it for
actually creating the server later.

The integration will happen in a following cl.
	Change on 2014/12/05 by yangg <yangg@google.com>
-------------
Fixed bugs in new_grpc_docker_builder.sh
	Change on 2014/12/05 by mlumish <mlumish@google.com>
-------------
In secure endpoint, hold a refcount for the life of a write callback if the
write does not complete immediately.
	Change on 2014/12/05 by klempner <klempner@google.com>
-------------
Add migration support to MOE and have TAP verify it doesn't break.

Migration support allows mirroring commits from [] into the git repo, instead of just a dump of the current source.
	Change on 2014/12/05 by ejona <ejona@google.com>
-------------
Change initial window size to 65535 according http2 draft 15.
	Change on 2014/12/05 by zhaoq <zhaoq@google.com>
-------------
Re-enable the flaky cases in dualstack_socket_test, with additional logging to
help track down the problem if it surfaces again.

This also seems like a good opportunity to make grpc_socket_utils a separate
library, as it's not really specific to TCP.

Example output:
logspam: [], 26570) resolved 2 addrs in 37ms:
logspam:   [0] [::1]:26570
logspam:   [1] 127.0.0.1:26570
	Change on 2014/12/05 by pmarks <pmarks@google.com>
-------------
Opensource build fixes.

-) A function that has a return type should actually return something.
-) Don't pass unsigned chars to strlen and strncmp.
	Change on 2014/12/05 by nnoble <nnoble@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81458281
2014-12-05 16:00:44 -08:00
vpai 8ac074ba20 Remove argument to grpc_em_alarm_cancel as it isn't used.
Change on 2014/12/01 by vpai <vpai@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81088661
2014-12-01 14:38:21 -08:00
Nicolas Noble b7ebd3b8c6 Initial import. 2014-11-26 16:33:03 -08:00