grpc/cpp/md_doc_xds-test-description...

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.17"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>GRPC C++: xDS (Load-Balancing) Interop Test Case Descriptions</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectalign" style="padding-left: 0.5em;">
   <div id="projectname">GRPC C++
   &#160;<span id="projectnumber">1.58.0</span>
   </div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.17 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
var searchBox = new SearchBox("searchBox", "search",false,'Search');
/* @license-end */
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
$(function() {
  initMenu('',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
/* @license-end */</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0"
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

</div><!-- top -->
<div class="PageDoc"><div class="header">
  <div class="headertitle">
<div class="title">xDS (Load-Balancing) Interop Test Case Descriptions </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>Client and server use <a href="../src/proto/grpc/testing/test.proto">test.proto</a>.</p>
<h1><a class="anchor" id="autotoc_md240"></a>
Server</h1>
<p>The code for the xDS test server can be found at: <a href="https://github.com/grpc/grpc-java/blob/master/interop-testing/src/main/java/io/grpc/testing/integration/XdsTestServer.java">Java</a> (other language implementations are in progress).</p>
<p>Server should accept these arguments:</p>
<ul>
<li>&ndash;port=PORT<ul>
<li>The port the test server will run on.</li>
</ul>
</li>
<li>&ndash;maintenance_port=PORT<ul>
<li>The port for the maintenance server running health, channelz, and admin(CSDS) services.</li>
</ul>
</li>
<li>&ndash;secure_mode=BOOLEAN<ul>
<li>When set to true it uses XdsServerCredentials with the test server for security test cases. In case of secure mode, port and maintenance_port should be different.</li>
</ul>
</li>
</ul>
<p>In addition, when handling requests, if the initial request metadata contains the <code>rpc-behavior</code> key, it should modify its handling of the request as follows:</p>
<ul>
<li>If the value matches <code>sleep-&lt;int&gt;</code>, the server should wait the specified number of seconds before resuming behavior matching and RPC processing.</li>
<li>If the value matches <code>keep-open</code>, the server should never respond to the request and behavior matching ends.</li>
<li>If the value matches <code>error-code-&lt;int&gt;</code>, the server should respond with the specified status code and behavior matching ends.</li>
<li>If the value matches <code>success-on-retry-attempt-&lt;int&gt;</code>, and the value of the <code>grpc-previous-rpc-attempts</code> metadata field is equal to the specified number, the normal RPC processing should resume and behavior matching ends.</li>
<li>A value can have a prefix <code>hostname=&lt;string&gt;</code> followed by a space. In that case, the rest of the value should only be applied if the specified hostname matches the server's hostname.</li>
</ul>
<p>The <code>rpc-behavior</code> header value can have multiple options separated by commas. In that case, the value should be split by commas and the options should be applied in the order specified. If a request has multiple <code>rpc-behavior</code> metadata values, each one should be processed that way in order.</p>
<h1><a class="anchor" id="autotoc_md241"></a>
Client</h1>
<p>The base behavior of the xDS test client is to send a constant QPS of unary messages and record the remote-peer distribution of the responses. Further, the client must expose an implementation of the <code>LoadBalancerStatsService</code> gRPC service to allow the test driver to validate the load balancing behavior for a particular test case (see below for more details).</p>
<p>The code for the xDS test client can be at: <a href="https://github.com/grpc/grpc-java/blob/master/interop-testing/src/main/java/io/grpc/testing/integration/XdsTestClient.java">Java</a> (other language implementations are in progress).</p>
<p>Clients should accept these arguments:</p>
<ul>
<li>&ndash;fail_on_failed_rpcs=BOOL<ul>
<li>If true, the client should exit with a non-zero return code if any RPCs fail after at least one RPC has succeeded, indicating a valid xDS config was received. This accounts for any startup-related delays in receiving an initial config from the load balancer. Default is false.</li>
</ul>
</li>
<li>&ndash;num_channels=CHANNELS<ul>
<li>The number of channels to create to the server.</li>
</ul>
</li>
<li>&ndash;qps=QPS<ul>
<li>The QPS per channel.</li>
</ul>
</li>
<li>&ndash;server=HOSTNAME:PORT<ul>
<li>The server host to connect to. For example, "localhost:8080"</li>
</ul>
</li>
<li>&ndash;stats_port=PORT<ul>
<li>The port for to expose the client's <code>LoadBalancerStatsService</code> implementation.</li>
</ul>
</li>
<li>&ndash;rpc_timeout_sec=SEC<ul>
<li>The timeout to set on all outbound RPCs. Default is 20.</li>
</ul>
</li>
<li>&ndash;secure_mode=BOOLEAN<ul>
<li>When set to true it uses XdsChannelCredentials with the test client for security test cases.</li>
</ul>
</li>
</ul>
<h2><a class="anchor" id="autotoc_md242"></a>
XdsUpdateClientConfigureService</h2>
<p>The xDS test client's behavior can be dynamically changed in the middle of tests. This is achieved by invoking the <code>XdsUpdateClientConfigureService</code> gRPC service on the test client. This can be useful for tests requiring special client behaviors that are not desirable at test initialization and client warmup. The service is defined as:</p>
<div class="fragment"><div class="line">message ClientConfigureRequest {</div>
<div class="line">  // Type of RPCs to send.</div>
<div class="line">  enum RpcType {</div>
<div class="line">    EMPTY_CALL = 0;</div>
<div class="line">    UNARY_CALL = 1;</div>
<div class="line">  }</div>
<div class="line"> </div>
<div class="line">  // Metadata to be attached for the given type of RPCs.</div>
<div class="line">  message Metadata {</div>
<div class="line">    RpcType type = 1;</div>
<div class="line">    string key = 2;</div>
<div class="line">    string value = 3;</div>
<div class="line">  }</div>
<div class="line"> </div>
<div class="line">  // The types of RPCs the client sends.</div>
<div class="line">  repeated RpcType types = 1;</div>
<div class="line">  // The collection of custom metadata to be attached to RPCs sent by the client.</div>
<div class="line">  repeated Metadata metadata = 2;</div>
<div class="line">  // The deadline to use, in seconds, for all RPCs.  If unset or zero, the</div>
<div class="line">  // client will use the default from the command-line.</div>
<div class="line">  int32 timeout_sec = 3;</div>
<div class="line">}</div>
<div class="line"> </div>
<div class="line">message ClientConfigureResponse {}</div>
<div class="line"> </div>
<div class="line">service XdsUpdateClientConfigureService {</div>
<div class="line">  // Update the tes client&#39;s configuration.</div>
<div class="line">  rpc Configure(ClientConfigureRequest) returns (ClientConfigureResponse);</div>
<div class="line">}</div>
</div><!-- fragment --><p>The test client changes its behavior right after receiving the <code>ClientConfigureRequest</code>. Currently it only supports configuring the type(s) of RPCs sent by the test client, metadata attached to each type of RPCs, and the timeout.</p>
<h1><a class="anchor" id="autotoc_md243"></a>
Test Driver</h1>
<p>Note that, unlike our other interop tests, neither the client nor the server has any notion of which of the following test scenarios is under test. Instead, a separate test driver is responsible for configuring the load balancer and the server backends, running the client, and then querying the client's <code>LoadBalancerStatsService</code> to validate load balancer behavior for each of the tests described below.</p>
<h1><a class="anchor" id="autotoc_md244"></a>
LoadBalancerStatsService</h1>
<p>The service is defined as:</p>
<div class="fragment"><div class="line">message LoadBalancerStatsRequest {</div>
<div class="line">  // Request stats for the next num_rpcs sent by client.</div>
<div class="line">  int32 num_rpcs = 1;</div>
<div class="line">  // If num_rpcs have not completed within timeout_sec, return partial results.</div>
<div class="line">  int32 timeout_sec = 2;</div>
<div class="line">}</div>
<div class="line"> </div>
<div class="line">message LoadBalancerStatsResponse {</div>
<div class="line">  message RpcsByPeer {</div>
<div class="line">    // The number of completed RPCs for each peer.</div>
<div class="line">    map&lt;string, int32&gt; rpcs_by_peer = 1;</div>
<div class="line">  }</div>
<div class="line">  // The number of completed RPCs for each peer.</div>
<div class="line">  map&lt;string, int32&gt; rpcs_by_peer = 1;</div>
<div class="line">  // The number of RPCs that failed to record a remote peer.</div>
<div class="line">  int32 num_failures = 2;</div>
<div class="line">  map&lt;string, RpcsByPeer&gt; rpcs_by_method = 3;</div>
<div class="line">}</div>
<div class="line"> </div>
<div class="line">message LoadBalancerAccumulatedStatsRequest {}</div>
<div class="line"> </div>
<div class="line">message LoadBalancerAccumulatedStatsResponse {</div>
<div class="line">  // The total number of RPCs have ever issued for each type.</div>
<div class="line">  // Deprecated: use stats_per_method.rpcs_started instead.</div>
<div class="line">  map&lt;string, int32&gt; num_rpcs_started_by_method = 1 [deprecated = true];</div>
<div class="line">  // The total number of RPCs have ever completed successfully for each type.</div>
<div class="line">  // Deprecated: use stats_per_method.result instead.</div>
<div class="line">  map&lt;string, int32&gt; num_rpcs_succeeded_by_method = 2 [deprecated = true];</div>
<div class="line">  // The total number of RPCs have ever failed for each type.</div>
<div class="line">  // Deprecated: use stats_per_method.result instead.</div>
<div class="line">  map&lt;string, int32&gt; num_rpcs_failed_by_method = 3 [deprecated = true];</div>
<div class="line"> </div>
<div class="line">  message MethodStats {</div>
<div class="line">    // The number of RPCs that were started for this method.</div>
<div class="line">    int32 rpcs_started = 1;</div>
<div class="line"> </div>
<div class="line">    // The number of RPCs that completed with each status for this method.  The</div>
<div class="line">    // key is the integral value of a google.rpc.Code; the value is the count.</div>
<div class="line">    map&lt;int32, int32&gt; result = 2;</div>
<div class="line">  }</div>
<div class="line"> </div>
<div class="line">  // Per-method RPC statistics.  The key is the RpcType in string form; e.g.</div>
<div class="line">  // &#39;EMPTY_CALL&#39; or &#39;UNARY_CALL&#39;</div>
<div class="line">  map&lt;string, MethodStats&gt; stats_per_method = 4;</div>
<div class="line">}</div>
<div class="line"> </div>
<div class="line">service LoadBalancerStatsService {</div>
<div class="line">  // Gets the backend distribution for RPCs sent by a test client.</div>
<div class="line">  rpc GetClientStats(LoadBalancerStatsRequest)</div>
<div class="line">      returns (LoadBalancerStatsResponse) {}</div>
<div class="line">  // Gets the accumulated stats for RPCs sent by a test client.</div>
<div class="line">  rpc GetClientAccumulatedStats(LoadBalancerAccumulatedStatsRequest)</div>
<div class="line">      returns (LoadBalancerAccumulatedStatsResponse) {}</div>
<div class="line">}</div>
</div><!-- fragment --><p>Note that the <code>LoadBalancerStatsResponse</code> contains the remote peer distribution of the next <code>num_rpcs</code> <em>sent</em> by the client after receiving the <code>LoadBalancerStatsRequest</code>. It is important that the remote peer distribution be recorded for a block of consecutive outgoing RPCs, to validate the intended distribution from the load balancer, rather than just looking at the next <code>num_rpcs</code> responses received from backends, as different backends may respond at different rates.</p>
<h1><a class="anchor" id="autotoc_md245"></a>
Test Cases</h1>
<h2><a class="anchor" id="autotoc_md246"></a>
ping_pong</h2>
<p>This test verifies that every backend receives traffic.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<ol type="1">
<li>&ndash;fail_on_failed_rpc=true</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>4 backends are created in a single managed instance group (MIG).</li>
</ol>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends receive at least one RPC</li>
</ol>
<h2><a class="anchor" id="autotoc_md247"></a>
round_robin</h2>
<p>This test verifies that RPCs are evenly routed according to an unweighted round robin policy.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<ol type="1">
<li>&ndash;fail_on_failed_rpc=true</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>4 backends are created in a single MIG.</li>
</ol>
<p>Test driver asserts that:</p>
<ol type="1">
<li>Once all backends receive at least one RPC, the following 100 RPCs are evenly distributed across the 4 backends.</li>
</ol>
<h2><a class="anchor" id="autotoc_md248"></a>
backends_restart</h2>
<p>This test verifies that the load balancer will resume sending traffic to a set of backends that is stopped and then resumed.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>4 backends are created in a single MIG.</li>
</ol>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends receive at least one RPC.</li>
</ol>
<p>The test driver records the peer distribution for a subsequent block of 100 RPCs then stops the backends.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>No RPCs from the client are successful.</li>
</ol>
<p>The test driver resumes the backends.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>Once all backends receive at least one RPC, the distribution for a block of 100 RPCs is the same as the distribution recorded prior to restart.</li>
</ol>
<h2><a class="anchor" id="autotoc_md249"></a>
secondary_locality_gets_requests_on_primary_failure</h2>
<p>This test verifies that backends in a secondary locality receive traffic when all backends in the primary locality fail.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>The primary MIG with 2 backends in the same zone as the client</li>
</ol>
<ol type="1">
<li>The secondary MIG with 2 backends in a different zone</li>
</ol>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends in the primary locality receive at least 1 RPC.</li>
</ol>
<ol type="1">
<li>No backends in the secondary locality receive RPCs.</li>
</ol>
<p>The test driver stops the backends in the primary locality.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends in the secondary locality receive at least 1 RPC.</li>
</ol>
<p>The test driver resumes the backends in the primary locality.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends in the primary locality receive at least 1 RPC.</li>
</ol>
<ol type="1">
<li>No backends in the secondary locality receive RPCs.</li>
</ol>
<h2><a class="anchor" id="autotoc_md250"></a>
secondary_locality_gets_no_requests_on_partial_primary_failure</h2>
<p>This test verifies that backends in a failover locality do not receive traffic when at least one of the backends in the primary locality remain healthy.</p>
<p><b>Note:</b> Future TD features may change the expected behavior and require changes to this test case.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>The primary MIG with 2 backends in the same zone as the client</li>
</ol>
<ol type="1">
<li>The secondary MIG with 2 backends in a different zone</li>
</ol>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends in the primary locality receive at least 1 RPC.</li>
</ol>
<ol type="1">
<li>No backends in the secondary locality receive RPCs.</li>
</ol>
<p>The test driver stops one of the backends in the primary locality.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends in the primary locality receive at least 1 RPC.</li>
</ol>
<ol type="1">
<li>No backends in the secondary locality receive RPCs.</li>
</ol>
<h2><a class="anchor" id="autotoc_md251"></a>
remove_instance_group</h2>
<p>This test verifies that a remaining instance group can successfully serve RPCs after removal of another instance group in the same zone.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>Two MIGs with two backends each, using rate balancing mode.</li>
</ol>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends receive at least one RPC.</li>
</ol>
<p>The test driver removes one MIG.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>All RPCs are directed to the two remaining backends (no RPC failures).</li>
</ol>
<h2><a class="anchor" id="autotoc_md252"></a>
change_backend_service</h2>
<p>This test verifies that the backend service can be replaced and traffic routed to the new backends.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<ol type="1">
<li>&ndash;fail_on_failed_rpc=true</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>One MIG with two backends</li>
</ol>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends receive at least one RPC.</li>
</ol>
<p>The test driver creates a new backend service containing a MIG with two backends and changes the TD URL map to point to this new backend service.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>All RPCs are directed to the new backend service.</li>
</ol>
<h2><a class="anchor" id="autotoc_md253"></a>
traffic_splitting</h2>
<p>This test verifies that the traffic will be distributed between backend services with the correct weights when route action is set to weighted backend services.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>One MIG with one backend</li>
</ol>
<p>Assert:</p>
<ol type="1">
<li>Once all backends receive at least one RPC, the following 1000 RPCs are all sent to MIG_a.</li>
</ol>
<p>The test driver adds a new MIG with 1 backend, and changes the route action to weighted backend services with {a: 20, b: 80}.</p>
<p>Assert:</p>
<ol type="1">
<li>Once all backends receive at least one RPC, the following 1000 RPCs are distributed across the 2 backends as a: 20, b: 80. </li>
</ol>
<h2><a class="anchor" id="autotoc_md254"></a>
path_matching</h2>
<p>This test verifies that the traffic for a certain RPC can be routed to a specific cluster based on the RPC path.</p>
<p>Client parameters:</p>
<ol type="1">
<li>–num_channels=1</li>
</ol>
<ol type="1">
<li>–qps=10</li>
</ol>
<ol type="1">
<li>–fail_on_failed_rpc=true</li>
</ol>
<ol type="1">
<li>–rpc=“EmptyCall,UnaryCall”</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>2 MIGs, each with 1 backend</li>
</ol>
<ol type="1">
<li>routes<ul>
<li>“/”: MIG_default</li>
</ul>
</li>
</ol>
<p>Assert:</p>
<ol type="1">
<li>UnaryCall RPCs are sent to MIG_default</li>
</ol>
<ol type="1">
<li>EmptyCall RPCs are sent to MIG_default</li>
</ol>
<p>The test driver changes route and asserts RPCs are sent to expected backends. <b>Note</b> that the default route <code>"/"</code> is always pointing to MIG_default, so all RPCs not matching the new route will be sent to MIG_default.</p>
<ul>
<li>{path: <code>/grpc.testing.TestService/EmptyCall</code>}: MIG_2<ul>
<li>UnaryCall -&gt; MIG_default</li>
<li>EmptyCall -&gt; MIG_2</li>
</ul>
</li>
<li>{prefix: <code>/grpc.testing.TestService/Unary</code>}: MIG_2<ul>
<li>UnaryCall -&gt; MIG_2</li>
<li>EmptyCall -&gt; MIG_default</li>
</ul>
</li>
<li>{prefix: <code>/grpc.testing.TestService/Unary</code>}: MIG_default &amp; {path: <code>/grpc.testing.TestService/EmptyCall</code>}: MIG_2<ul>
<li>UnaryCall -&gt; MIG_default</li>
<li>EmptyCall -&gt; MIG_2</li>
</ul>
</li>
<li>{regex: <code>^\/.*\/UnaryCall$</code>}: MIG_2<ul>
<li>UnaryCall -&gt; MIG_2</li>
<li>EmptyCall -&gt; MIG_default</li>
</ul>
</li>
<li>{path: <code>/gRpC.tEsTinG.tEstseRvice/empTycaLl</code>, ignoreCase: <code>True</code>}: MIG_2<ul>
<li>UnaryCall -&gt; MIG_default</li>
<li>EmptyCall -&gt; MIG_2</li>
</ul>
</li>
</ul>
<h2><a class="anchor" id="autotoc_md255"></a>
header_matching</h2>
<p>This test verifies that the traffic for a certain RPC can be routed to a specific cluster based on the RPC header (metadata).</p>
<p>Client parameters:</p>
<ol type="1">
<li>–num_channels=1</li>
</ol>
<ol type="1">
<li>–qps=10</li>
</ol>
<ol type="1">
<li>–fail_on_failed_rpc=true</li>
</ol>
<ol type="1">
<li>–rpc=“EmptyCall,UnaryCall”</li>
</ol>
<ol type="1">
<li>–rpc=“EmptyCall:xds_md:exact_match”</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>2 MIGs, each with 1 backend</li>
</ol>
<ol type="1">
<li>routes<ul>
<li>“/”: MIG_default</li>
</ul>
</li>
</ol>
<p>Assert:</p>
<ol type="1">
<li>UnaryCall RPCs are sent to MIG_default</li>
</ol>
<ol type="1">
<li>EmptyCall RPCs are sent to MIG_default</li>
</ol>
<p>The test driver changes route and asserts RPCs are sent to expected backends. <b>Note</b> that the default route <code>"/"</code> is always pointing to MIG_default, so all RPCs not matching the new route will be sent to MIG_default.</p>
<ul>
<li>{header <code>xds_md</code>, exact: <code>empty_ytpme</code>}: MIG_2<ul>
<li>Unary -&gt; MIG_default</li>
<li>Empty -&gt; MIG_2</li>
</ul>
</li>
<li>{header <code>xds_md</code>, prefix: <code>un</code>}: MIG_2<ul>
<li><code>un</code> is the prefix of metadata sent with UnaryCall</li>
<li>Unary -&gt; MIG_2</li>
<li>Empty -&gt; MIG_default</li>
</ul>
</li>
<li>{header <code>xds_md</code>, suffix: <code>me</code>}: MIG_2<ul>
<li><code>me</code> is the suffix of metadata sent with EmptyCall</li>
<li>Unary -&gt; MIG_default</li>
<li>Empty to MIG_2</li>
</ul>
</li>
<li>{header <code>xds_md_numeric</code>, present: <code>True</code>}: MIG_2<ul>
<li>Unary is sent with the metadata, so will be sent to alternative</li>
<li>Unary -&gt; MIG_2</li>
<li>Empty -&gt; MIG_default</li>
</ul>
</li>
<li>{header <code>xds_md</code>, exact: <code>unary_yranu</code>, invert: <code>True</code>}: MIG_2<ul>
<li>Unary is sent with the metadata, so this will not match Unary, but will match Empty</li>
<li>Unary -&gt; MIG_default</li>
<li>Empty to MIG_2</li>
</ul>
</li>
<li>{header <code>xds_md_numeric</code>, range <code>[100,200]</code>}: MIG_2<ul>
<li>Unary is sent with the metadata in range</li>
<li>Unary -&gt; MIG_2</li>
<li>Empty -&gt; MIG_default</li>
</ul>
</li>
<li>{header <code>xds_md</code>, regex: <code>^em.*me$</code>}: MIG_2<ul>
<li>EmptyCall is sent with the metadata</li>
<li>Unary -&gt; MIG_default</li>
<li>Empty -&gt; MIG_2</li>
</ul>
</li>
</ul>
<h2><a class="anchor" id="autotoc_md256"></a>
gentle_failover</h2>
<p>This test verifies that traffic is partially diverted to a secondary locality when &gt; 50% of the instances in the primary locality are unhealthy.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>The primary MIG with 3 backends in the same zone as the client</li>
</ol>
<ol type="1">
<li>The secondary MIG with 2 backends in a different zone</li>
</ol>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends in the primary locality receive at least 1 RPC.</li>
</ol>
<ol type="1">
<li>No backends in the secondary locality receive RPCs.</li>
</ol>
<p>The test driver stops 2 of 3 backends in the primary locality.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends in the secondary locality receive at least 1 RPC.</li>
</ol>
<ol type="1">
<li>The remaining backend in the primary locality receives at least 1 RPC.</li>
</ol>
<p>The test driver resumes the backends in the primary locality.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends in the primary locality receive at least 1 RPC.</li>
</ol>
<ol type="1">
<li>No backends in the secondary locality receive RPCs.</li>
</ol>
<h2><a class="anchor" id="autotoc_md257"></a>
load_based_failover</h2>
<p>This test verifies that traffic is partially diverted to a secondary locality when the QPS is greater than the configured RPS in the priority locality.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>The primary MIG with 2 backends in the same zone as the client</li>
</ol>
<ol type="1">
<li>The secondary MIG with 2 backends in a different zone</li>
</ol>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends in the primary locality receive at least 1 RPC.</li>
</ol>
<ol type="1">
<li>No backends in the secondary locality receive RPCs.</li>
</ol>
<p>The test driver sets <code>balancingMode</code> is <code>RATE</code>, and <code>maxRate</code> to 20 in the primary locality.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends in the primary locality receive at least 1 RPC.</li>
</ol>
<ol type="1">
<li>All backends in the secondary locality receive at least 1 RPC.</li>
</ol>
<p>The test driver set <code>maxRate</code> to 120 in the primary locality.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>All backends in the primary locality receive at least 1 RPC.</li>
</ol>
<ol type="1">
<li>No backends in the secondary locality receive RPCs.</li>
</ol>
<h2><a class="anchor" id="autotoc_md258"></a>
circuit_breaking</h2>
<p>This test verifies that the maximum number of outstanding requests is limited by circuit breakers of the backend service.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>Two MIGs with each having two backends.</li>
</ol>
<p>The test driver configures the backend services with:</p>
<ol type="1">
<li>path{“/grpc.testing.TestService/UnaryCall"}: MIG_1
1. path{“/grpc.testing.TestService/EmptyCall"}: MIG_2</li>
</ol>
<ol type="1">
<li>MIG_1 circuit_breakers with max_requests = 500</li>
</ol>
<ol type="1">
<li>MIG_2 circuit breakers with max_requests = 1000</li>
</ol>
<p>The test driver configures the test client to send both UnaryCall and EmptyCall, with all RPCs keep-open.</p>
<p>Assert:</p>
<ol type="1">
<li>After reaching steady state, there are 500 UnaryCall RPCs in-flight and 1000 EmptyCall RPCs in-flight.</li>
</ol>
<p>The test driver updates MIG_1's circuit breakers with max_request = 800.</p>
<p>Test driver asserts:</p>
<ol type="1">
<li>After reaching steady state, there are 800 UnaryCall RPCs in-flight.</li>
</ol>
<h2><a class="anchor" id="autotoc_md259"></a>
timeout</h2>
<p>This test verifies that traffic along a route with a <code>max_stream_duration</code> set will cause timeouts on streams open longer than that duration.</p>
<p>Client parameters:</p>
<ol type="1">
<li><code>--num_channels=1</code></li>
</ol>
<ol type="1">
<li><code>--qps=100</code></li>
</ol>
<p>Route Configuration:</p>
<p>Two routes:</p>
<ol type="1">
<li>Path match for <code>/grpc.testing.TestService/UnaryCall</code>, with a <code>route_action</code> containing <code>max_stream_duration</code> of 3 seconds.</li>
</ol>
<ol type="1">
<li>Default route containing no <code>max_stream_duration</code> setting.</li>
</ol>
<p>There are four sub-tests:</p>
<ol type="1">
<li><code>app_timeout_exceeded</code><ol type="a">
<li>Test client configured to send UnaryCall RPCs with a 1s application timeout, and metadata of <code>rpc-behavior: sleep-2</code>.</li>
</ol>
<ol type="a">
<li>Test driver asserts client recieves ~100% status <code>DEADLINE_EXCEEDED</code>.</li>
</ol>
</li>
</ol>
<ol type="1">
<li><code>timeout_not_exceeded</code><ol type="a">
<li>Test client configured to send UnaryCall RPCs with the default application timeout (20 seconds), and no metadata.</li>
</ol>
<ol type="a">
<li>Test driver asserts client recieves ~100% status <code>OK</code>.</li>
</ol>
</li>
</ol>
<ol type="1">
<li><code>timeout_exceeded</code> (executed with the below test case)</li>
</ol>
<ol type="1">
<li><code>timeout_different_route</code><ol type="a">
<li>Test client configured to send UnaryCall RPCs and EmptyCall RPCs with the default application timeout (20 seconds), and metadata of <code>rpc-behavior: sleep-4</code>.</li>
</ol>
<ol type="a">
<li>Test driver asserts client recieves ~100% status <code>OK</code> for EmptyCall and ~100% status <code>DEADLINE_EXCEEDED</code> for UnaryCall.</li>
</ol>
</li>
</ol>
<h2><a class="anchor" id="autotoc_md260"></a>
api_listener</h2>
<p>The test case verifies a specific use case where it creates a second TD API listener using the same name as the existing one and then delete the old one. The test driver verifies this is a safe way to update the API listener configuration while keep using the existing name.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>One MIG with two backends.</li>
</ol>
<p>Assert:</p>
<p>The test driver configuration steps:</p><ol type="1">
<li>The test driver creates the first set of forwarding rule + target proxy + URL map with a test host name.</li>
</ol>
<ol type="1">
<li>Then the test driver creates a second set of forwarding rule + target proxy + URL map with the same test host name.</li>
</ol>
<ol type="1">
<li>The test driver deletes the first set of configurations in step 1.</li>
</ol>
<p>The test driver verifies, at each configuration step, the traffic is always able to reach the designated hosts.</p>
<h2><a class="anchor" id="autotoc_md261"></a>
metadata_filter</h2>
<p>This test case verifies that metadata filter configurations in URL map match rule are effective at Traffic Director for routing selection against downstream node metadata.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>Two MIGs in the same zone, each having two backends.</li>
</ol>
<p>There are four test sub-cases:</p><ol type="1">
<li>Test <code>MATCH_ALL</code> metadata filter criteria.</li>
</ol>
<ol type="1">
<li>Test <code>MATCH_ANY</code> metadata filter criteria.</li>
</ol>
<ol type="1">
<li>Test mixed <code>MATCH_ALL</code> and <code>MATCH_ANY</code> metadata filter criteria.</li>
</ol>
<ol type="1">
<li>Test when multiple match rules with metadata filter all match.</li>
</ol>
<p>Assert:</p>
<p>At each test sub-case described above, the test driver configures and verifies:</p>
<ol type="1">
<li>Set default URL map, and verify traffic goes to the original backend hosts.</li>
</ol>
<ol type="1">
<li>Then patch URL map to update the match rule with metadata filter configuration under test added.</li>
</ol>
<ol type="1">
<li>Then it verifies traffic switches to alternate backend service hosts.</li>
</ol>
<p>This way, we test that TD correctly evaluates both matching and non-matching configuration scenario.</p>
<h2><a class="anchor" id="autotoc_md262"></a>
forwarding_rule_port_match</h2>
<p>This test verifies that request server uri port should match with the GCP forwarding rule configuration port.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>One MIG with two backends.</li>
</ol>
<p>Assert:</p><ol type="1">
<li>The test driver configures matching port in the forwarding rule and in the request server uri, then verifies traffic reaches backend service instances.</li>
</ol>
<ol type="1">
<li>The test driver updates the forwarding rule to use a different port, then verifies that the traffic stops going to those backend service instances.</li>
</ol>
<h2><a class="anchor" id="autotoc_md263"></a>
forwarding_rule_default_port</h2>
<p>This test verifies that omitting port in the request server uri should only match with the default port(80) configuration in the forwarding rule. In addition, request server uri port should exactly match that in the URL map host rule, as described in <a href="https://cloud.google.com/traffic-director/docs/proxyless-overview#proxyless-url-map">public doc</a>.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
</ol>
<ol type="1">
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>One MIG with two backends.</li>
</ol>
<p>Assert:</p>
<p>Test driver configures and verifies:</p><ol type="1">
<li>No traffic goes to backends when configuring the target URI <code>xds:///myservice</code>, the forwarding rule with port <em>x != 80</em>, the URL map host rule <code>myservice::x</code>.</li>
</ol>
<ol type="1">
<li>Traffic goes to backends when configuring the target URI <code>xds:///myservice</code>, the forwarding rule port <code>80</code> and the URL map host rule <code>myservice</code>.</li>
</ol>
<ol type="1">
<li>No traffic goes to backends when configuring the target URI <code>xds:///myservice</code>, the forwarding rule port <code>80</code> and the host rule <code>myservice::80</code>.</li>
</ol>
<h2><a class="anchor" id="autotoc_md264"></a>
outlier_detection</h2>
<p>This test verifies that the client applies the outlier detection configuration and temporarily drops traffic to a server that fails requests.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<ol type="1">
<li>One MIG with five backends, with a <code>backendService</code> configuration with the following <code>outlierDetection</code> entry ```json { "interval": { "seconds": 2, "nanos": 0 }, "successRateRequestVolume": 20 } ``` Assert:</li>
</ol>
<ol type="1">
<li>The test driver asserts that traffic is equally distribted among the five backends, and all requests end with the <code>OK</code> status.</li>
<li>The test driver chooses one of the five backends to fail requests, and configures the client to send the metadata <code>rpc-behavior: hostname=&lt;chosen backend&gt; error-code-2</code>. The driver asserts that during some 10-second interval, all traffic goes to the other four backends and all requests end with the <code>OK</code> status.</li>
<li>The test driver removes the client configuration to send metadata. The driver asserts that during some 10-second interval, traffic is equally distributed among the five backends, and all requests end with the <code>OK</code> status.</li>
</ol>
<h2><a class="anchor" id="autotoc_md265"></a>
custom_lb</h2>
<p>This test verifies that a custom load balancer policy can be configured in the client. It also verifies that when given a list of policies the client can ignore a bad one and try the next one on the list until it finds a good one.</p>
<p>Client parameters:</p>
<ol type="1">
<li>&ndash;num_channels=1</li>
<li>&ndash;qps=100</li>
</ol>
<p>Load balancer configuration:</p>
<p>One MIG with a single backend.</p>
<p>The <code>backendService</code> will have the following <code>localityLbPolicies</code> entry: </p><div class="fragment"><div class="line">[ </div>
<div class="line">  {</div>
<div class="line">    &quot;customPolicy&quot;: {</div>
<div class="line">      &quot;name&quot;: &quot;test.ThisLoadBalancerDoesNotExist&quot;,</div>
<div class="line">      &quot;data&quot;: &quot;{ \&quot;foo\&quot;: \&quot;bar\&quot; }&quot;</div>
<div class="line">    }</div>
<div class="line">  },</div>
<div class="line">  {</div>
<div class="line">    &quot;customPolicy&quot;: {</div>
<div class="line">      &quot;name&quot;: &quot;test.RpcBehaviorLoadBalancer&quot;,</div>
<div class="line">      &quot;data&quot;: &quot;{ \&quot;rpcBehavior\&quot;: \&quot;error-code-15\&quot; }&quot;</div>
<div class="line">    }</div>
<div class="line">  }</div>
<div class="line">]</div>
</div><!-- fragment --><p>The client <b>should not</b> implement the <code>test.ThisLoadBalancerDoesNotExist</code>, but it <b>should</b> implement <code>test.RpcBehaviorLoadBalancer</code>. The <code>RpcBehaviorLoadBalancer</code> implementation should set the rpcBehavior request header based on the configuration it is provided. The <code>rpcBehavior</code> field value in the config should be used as the header value.</p>
<p>Assert:</p>
<ol type="1">
<li>The first custom policy is ignored as the client does not have an implementation for it.</li>
<li>The second policy, that <b>is</b> implemented by the client, has been applied by the client. This can be asserted by confirming that each request has failed with the configured error code 15 (DATA_LOSS). We should get this error because the test server knows to look for the <code>rpcBehavior</code> header and fail a request with a provided error code.</li>
</ol>
<p>Note that while this test is for load balancing, we can get by with a single backend as our test load balancer does not perform any actual load balancing, instead only applying the <code>rpcBehavior</code> header to each request. </p>
</div></div><!-- contents -->
</div><!-- PageDoc -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Wed Sep 6 2023 16:09:16 for GRPC C++ by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/>
</a> 1.8.17
</small></address>
</body>
</html>